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Abstract 


To  ensure  the  capability  of  defence,  a  demand  for  equipment  and  systems  which  can  be  embraced  under  the  title  of  “Robotics” 
will  emerge  in  the  near  future.  In  this  context,  “Robotics”  represents  a  specific  problem  area  involving  all  the  guidance  and 
control  functions  which  are  associated  with  achieving  goal-oriented  autonomous  behaviour  in  structured  and  unstructured 
environments  for  mobile  and  manipulator  systems  as  applied  to  ground,  sea,  air  and  space  operations.  Related  robotic  systems 
must  combine  constituent  functions  such  as  intelligent  decision  making,  control,  manipulation,  motion,  sensing  and 
communication. 

The  scope  of  the  special  course  will  cover  new  developments  in  the  areas  of  autonomous  navigation  for  planetary  and  surface 
systems,  and  control  and  operations  of  remote  manipulators. 

Topics  to  be  covered  include: 

•  Kinematics,  dynamics  and  mobility; 

•  Sensing  (vision,  tactile,  acoustic,  etc.)  and  sensory  processing; 

•  Sensory  interactive  task  decomposition,  planning  and  problem  solving; 

•  World  modelling; 

•  Programming  techniques  and  learning,  cognitive  control,  adaptive  sensory-motor  control; 

•  System  integration,  test  and  evaluation; 

•  Man-machine  interfaces. 


Abrege 


Pour  garantir  les  capacites  de  defense,  un  besoin  urgent  en  equipements  et  systemes  qu’il  convient  de  regrouper  sous  le  vocable 
de  “robotique”  se  fera  sentir  dans  un  futur  proche. 

Dans  ce  contexte,  la  robotique  ouvre  un  domaine  de  problemes  specifiques  mettant  en  oeuvre  toutes  les  fonctions  de  guidage  et 
de  pilotage  qui  sont  associees  au  fonctionnement  autonome  sur  objectif  defini,  dans  des  environnements  structures  ou  non,  des 
systemes  mobiles  ou  des  manipulateurs  impliques  dans  des  operations  terrestres,  maritimes,  aeriennes  et  spatiales. 

Ces  systemes  robotiques  doivent  combiner  des  fonctions  composantes  telles  que  le  prise  de  decision  intelligente,  le  pilotage,  la 
manipulation,  le  mouvement,  la  detection  et  les  communications. 

Ce  corns  special  presentera  les  nouveaux  developpements  dans  les  domaines  de  la  navigation  autonome  pour  systemes 
planetaires  et  terrestres  ainsi  que  le  pilotage  et  le  fonctionnement  des  telemanipulateurs.  Les  sujets  examines  comprennent: 

•  la  cinematique,  la  dynamique  et  la  mobilite; 

•  la  detection  (visuelle,  tactile,  acoustique  etc.)  et  le  traitement  sensoriel; 

•  la  decomposition  des  taches  sensorielles  interactives,  la  planification  et  la  resolution  des  problemes; 

•  la  modelisation  du  monde; 

•  les  techniques  de  programmation  et  l’apprentissage,  le  controle  cognitif,  le  controle  sensoriel-moteur  adaptatif 

•  Tintegration  systemes,  les  essais  et  P  evaluation 

•  les  interfaces  homme-machine. 
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Robotics  Technology  Developments  in  the 
United  States  Space  Telerobotics  Program 


Dave  Lavery 

Telerobotics  Program  Manager 
Office  of  Advanced  Concepts  and  Technology 
National  Aeronautics  and  Space  Administration 
W  ashington,  DC  25046,  USA 


ABSTRACT 

In  the  same  way  that  the  launch  of 
Yuri  Gagarin  in  April  1961  announced 
the  beginning  of  human  space  flight, 
last  year’s  flight  of  the  German  ROTEX 
robot  flight  experiment  is  heralding  the 
start  of  a  new  era  of  space  robotics. 
After  a  gap  of  twelve  years  since  the 
introduction  of  a  new  capability  in 
space  remote  manipulation,  ROTEX  is 
the  first  of  at  least  ten  new  robotic 
systems  and  experiments  which  will 
fly  before  the  year  2000. 

As  a  result  of  redefining  the 
development  approach  for  space  robotic 
systems,  and  capitalizing  on 
opportunities  associated  with  the 
assembly  and  maintenance  of  the 
Space  Station,  the  space  robotics 
community  is  preparing  a  whole  new 
generation  of  operational  robotic 
capabilities.  Expanding  on  the 
capabilities  of  earlier  manipulation 
systems  such  as  the  Viking  and 
Surveyor  soil  scoops,  the  Russian 
Lunakhods,  and  the  Shuttle  Remote 
Manipulator  System  (RMS),  these  new 
space  robots  will  augment  astronaut 
on-orbit  capabilities  and  extend  virtual 
human  presence  to  lunar  and 
planetary  surfaces. 

BACKGROUND 

During  its  history,  NASA  has 
undertaken  a  number  of  research 
programs  aimed  at  developing  remote 
manipulation  capabilities  for  use  in 


space.  Extending  back  to  1961,  these 
programs  pursued  the  development  of 
technologies  for  remote  maniuplation 
with  and  without  force  feedback,  coping 
with  communication  time  delays, 
advanced  system  autonomy,  advanced 
control  techniques,  free-flying  and 
fixed  base  maniuplators,  roving 
vehicles,  intelligent  mechanisms, 
operator  interfaces,  sensor 
components,  control  execution, 
perception  systems,  and  mechatronics, 
among  others.1 

During  this  entire  time  NASA  and  the 
world-wide  aerospace  community 
gained  very  little  experience  in  the 
operational  use  of  robotic  manipulation 
systems  in  extra-terrestrial 
environments.  To  date,  there  have  been 
only  four  examples  of  a  remotely 
manipulated  device  being  used  as  an 
operational  component  of  a  spacecraft 
mission  in  an  extra-terrestrial 
environment.  This  set  is  limited  to: 

•  the  Surveyor  3,  5,  6,  and  7 
missions,  which  soft- 
landed  on  the  Lunar 
surface  in  1967  and  1968. 

The  Surveyors  used  a 
three-degree-of-freedom 
(DOF)  soil  scoop  used  to  dig 
small  trenches  in  the 
Lunar  regolith  in  the 
immediate  vicinity  of  the 
landing  site.  Samples  from 
the  trenches  were  then 
collected  by  the 
manipulator  and  placed 
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onboard  the  spacecraft  for 
analysis. 


Figure  1:  Surveyor  spacecraft  with 
deployed  sampling  manipulator 


the  Viking  1  and  Viking  2 
missions  which  landed  on 
Mars  in  July  and 
September  of  1976.  The 
Viking  landers  included  a 
four-DOF  soil  scoop  used  to 
collect  Mars  soil 
samples  in  the 
immediate  vicinity  of 
the  spacecraft. 
Samples  from  the 
trenches  were  then 
collected  by  the 
manipulator  and 
placed  onboard  for 
analysis. 

the  Soviet  Lunakhod 
missions,  which  sent 
two  unmanned  mobile 
wheeled  rover  vehicles 
to  the  Lunar  surface. 

Each  of  these  vehicles 
was  operated  by 
remote  teleoperation 
without  time-delay 
compensation. 
Navigation  of  the 


vehicles  was  performed  via 
a  observe-plan-transmit- 
move-wait  strategy. 
Lunakhod  1  operated  for 
approximately  eleven 
Lunar  days,  traveling  a 
total  of  10  kilometers. 
Lunakhod  2,  with  twice  the 
speed  of  Lunakhod  1  and 
more  experienced 
controllers,  traveled  35 
kilometers  in  about  five 
Lunar  days.  In  addition  to 
the  teleoperated  navigation 
capability,  each  Lunakhod 
also  utilized  a  teleoperated 
point  penetrometer  which 
measured  regolith 
densities. 

the  current  Space  Shuttle 
Remote  Manipulator 
System  (RMS),  which  was 
first  flown  aboard  STS-2  in 
1981.  This  six-DOF 
manipulator  is  a  50-foot 
long,  three-segment 


Figure  2:  Viking  spacecraft  with  deployed 
sampling  manipulator 
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teleoperator  with  a  snare- 
before-contact  end-effector, 
and  is  designed  for  the 
deployment,  capture  and 
recovery  of  large  Space 
Transportation  System 
(STS)  payloads.  The  RMS 
has  been  used  to  perform 
several  tasks  on-orbit, 
including  the  deployment 
of  the  Hubble  Space 
Telescope  (HST),  use  as  an 
EVA  astronaut  foot 
restraint  during  the  repair 
of  Westar  and  Palapa-B, 
knocking  ice  from  water 
dump  nozzles  on  STS-41-D, 
and  retrieval  of  the  21,300 
pound  Long  Duration 
Exposure  Facility  (LDEF). 


Figure  3:  STS  with  the  RMS  deploying  a 
payload 


In  all  cases  these  devices  are,  by 
terrestrial  standards,  fairly  primitive 
teleoperators  with  very  limited 
capabilities.  For  example,  terrestrial 
robots  inherently  have  many 
capabilities  which  allow  them  to 
outperform  humans  on  some  tasks. 
They  can  be  more  precise  and  more 
repeatable  and  do  not  become  tired  or 
bored.  Robots  are  chosen  to  accomplish 
tasks  which  are  highly  repetitive, 
which  are  very  well  defined,  which 
exist  in  stable,  unchanging 
environments,  and  do  not  require 
significant  oversight  or  human 
intervention.  Current  terrestrial 


robotics  can  be  characterized  as: 

•  limited  to  industrial 
manufacturing 

•  preprogrammed  control 

•  precisely  structured 
environments 

•  single,  highly  repetitive 
tasks 

•  heavy,  rigid  devices 

Teleoperators,  with  human  controllers, 
have  an  advantage  over  robots  in  some 
other  situations  because  of  the  difficulty 
of  preprogramming  reactions  to  all  of 
the  contingencies  that  may  occur 
during  a  task.  Current  terrestrial 
teleoperation  can  be  characterized  as: 

•  limited  to  undersea  and 
nuclear  applications 

•  limited  to  real-time 
manual  control 

•  used  in  semi-structured 
environments  that  are 
hostile  to  humans 

•  non-repetitive  tasks 

•  limited  maintainability  and 
considerable  backlash 

Terrestrially,  teleoperators  are 
currently  the  choice  for  tasks  which 
are  not  done  sufficiently  often  to 
amortize  the  cost  of  programming  a 
robot,  tasks  in  which  the  environment 
cannot  be  sufficiently  controlled  to 
permit  robot  operation,  tasks  in  which 
sufficient  manual  dexterity,  sensing, 
and  artificial  intelligence  is  not  yet 
available  in  robots,  and  tasks  in  which 
a  human  operator  is  warranted 
because  of  the  cost  of  a  possible  failure 
of  a  robot  is  too  high. 

By  comparison,  space  telerobotics 
technology  requirements  can  be 
characterized  by:  2 

•  need  for  both  manual  and 
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automated  control 

•  semi-  to  unstructured 
environments 

•  non-repetitive  tasks 

•  variable  time  delay  between 
operator  and  manipulator 

•  dexterous,  lightweight  and 
flexible  manipulators 

•  complex  kinematics  and 
dynamics 

•  novel  locomotion 
mechanisms 

•  minimal  and  simple 
servicing  of  the  device 

•  hostile  environment  of 
thermal  gradients, 
radiation,  vacuum, 
variable  lighting 

•  ability  to  recover  from 
unplanned  events, 
including  system  faults 
and  errors 

In  short,  the  next  generation  of  space 
telerobotics  must  be  far  more  flexible 
than  the  current  generations  of 
terrestrial  robots  and  teleoperators. 

Biting  Off  Too  Much 

Historically,  the  space  robotics 
community  has  been  pursuing  the  goal 
of  creating  fully  autonomous,  self- 
contained  robotic  systems  with 
considerable  onboard  intelligence  as 
the  next  major  objective  in  space 
robotics  evolution.  Systems  such  as  the 
Flight  Telerobotic  Servicer  (FTS)  were 
intended  to  provide  near-human  levels 
of  intelligence  and  dexterity,  capable  of 
interpreting  very  high  level  command 
structures  and  autonomously 
executing  the  commands  without 
human  intervention.  Incomplete  task 
operations,  erroneous  command 
structures,  and  reconciliation  of 
differences  between  the  world  model 
and  the  real  world  were  to  be  handled 


autonomously,  and  the  entire  system 
was  to  survive  for  up  to  30  years  on- 
orbit  with  little  or  no  maintenance. 
The  robot  was  designed  to  replace  a 
full-time  human  operator  with 
automated  sensing,  perception, 
planning  and  reasoning  sufficient  to 
conduct  daily  operations.  The  incentive 
was  to  remove  the  need  for  intensive 
human  oversight  of  the  robot’s 
activities,  and  (anticipating  that  the 
human  operator  would  be  an  on-orbit 
astronaut)  thereby  reduce  the 
astronaut  workload. 

Since  the  initiation  of  the  FTS  and 
similar  ambitious  undertakings,  the 
robotics  community  has  gained  new 
understanding  of  the  research  still 
required  to  create  the  technologies 
needed  for  such  systems.  While  very 
significant  progress  has  been  made  in 
supporting  technologies  such  as 
system  autonomy,  task  level 
controllers,  control  execution  and 
robust  operations,  the  day  when  all  of 
these  elements  will  come  together  into 
an  operational  autonomous  space  robot 
system  is  probably  still  a  decade  away 
(some  roboticists  have  conjectured  that 
it  will  not  happen  during  their 
professional  lifetimes).3 

A  New  Focus 

While  the  technology  to  support  fully 
autonomous  intelligent  robotics  is  not 
yet  available,  operational  needs  for 
capable  remote  manipulation  and 
locomotion  still  exist.  To  contend  with 
these  needs,  the  space  robotics 
community  has  adopted  a  new 
approach  which  parallels  that  of  the 
underwater  robotics  industry. 

Until  recently,  deep  subsea  robotic 
operations  were  conducted  with  small 
manned  submarines  where  the 
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operator  had  direct  local  control  over 
manipulators  physically  attached  to  the 
sub.  The  operator’s  interface  with  the 
manipulators  was  limited,  the 
workspace  was  cramped,  and  the 
environment  was  inherently 
dangerous.  Recent  advances  in  remote 
operation  technology  has  permitted  a 
shift  in  the  operational  paradigm, 
where  manipulators  are  now  carried 
on  unmanned  remotely  operated 
vehicles  (ROVs)  which  are  controlled 
by  remote  operators  safely  and 
(relatively)  comfortably  located  aboard  a 
surface  ship.  Passing  commands  to 
the  unmanned  ROV  via  an  umbilical, 
the  operator  utilizes  an  early  version  of 
telepresence  technology  to  visualize  the 
remote  scene  surrounding  the  ROV 
and  control  the  manipulators. 

In  the  space  robotics  arena,  this  same 
paradigm  shift  is  taking  place.  Rather 
than  attempting  to  force  the  use  of 
immature  technology  to  emulate  the 
“smarts”  of  a  local  astronaut  operator, 
the  new  focus  is  to  utilize  advanced 
teleoperation  technology  to  move  the 
operator  from  close  proximity  on-orbit 
to  the  ground.  Technology  elements 
including  predictive  displays,  low-level 
reactive  planners,  sensor-based 
command  execution  and  dynamic 
world  modeling  enable  the  ability  to 
contend  with  problems  associated  with 
relocation  of  the  operator,  such  as  time- 
delayed  communications,  limited 
viewing  options  and  limited  command 
stream  bandwidth.  While  there  is  still 
a  long-term  goal  of  developing 
intelligent  autonomy  for  robots,  the 
short-term  goal  has  become  the 
development  of  technology  to  push 
forward  “intelligent  teleoperation.” 

The  major  impact  of  this  shift  in 
development  philosophy  is  the  new 
opportunity  to  move  robotics  out  of  the 
laboratory  and  into  the  field.  The 


maturation  of  advanced  teleoperation 
technologies  has  helped  increase 
confidence  in  the  ability  of  robotic 
systems  to  robustly  perform  real  tasks. 
With  this  increased  confidence  has 
come  the  acceptance  of  the  potential 
benefits  offered  by  space  robotics 
technology,  and  the  challenge  to  “fly  it 
and  prove  it”  with  a  series  of  robotic 
flight  experiments  and 
demonstrations.  For  the  first  time 
since  the  first  flight  of  the  Shuttle  RMS, 
new  operational  robotic  tools  will  be 
seen  on-orbit  and  on  planetary 
surfaces.  And  rather  than  waiting  for 
more  than  a  decade  between  the 
introductions  of  new  robotic  systems, 
the  new  push  to  “get  things  flying”  will 
yield  multiple  new  space  robotic 
systems  before  the  end  of  the  1990’s. 

The  robotic  systems  to  be  flown  during 
the  next  five  years  fall  into  three 
distinct  categories:  Extra- Vehicular 
Robotic  (EVR)  servicers,  science 
payload  servicers,  and  planetary 
surface  rovers.  Flight  experiments, 
technology  demonstrations  and 
operational  systems  are  being  built  in 
each  of  these  categories  by  NASA, 
academic,  industrial,  and 
international  developers. 


EVR  Servicers 

The  EVR  servicer  systems  are  robotic 
systems  deployed  in  Earth  orbit  for  use 
outside  of  pressurized,  controlled 
environments.  Such  systems  are 
typified  by  the  Shuttle  RMS,  which  was 
first  flown  on  the  STS-2  mission  is  1981. 
Target  applications  for  these  systems 
include  on-orbit  satellite  assembly, 
maintenance,  repair  and  servicing, 
robotic  enhancement  of  Shuttle  payload 
bay  operations,  and  ground-control 
robotic  servicing  of  external  Space 
Station  payloads.  Within  the  EVR 
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servicers  class,  six  different  efforts  are 
currently  underway,  in  Space  Station- 
and  Shuttle-attached  as  well  as  free- 
flying  configurations. 

Canada  is  providing  two  space  robots 
for  use  on  the  International  Space 
Station.  The  Space  Station  Remote 
Manipulator  System  (SSRMS)  is  a  55- 
foot  long,  7-Degree  Of  Freedom  (7-DOF) 
manipulator  similar  to  the  Shuttle 
RMS.  Designed  to  maneuver  and 
locate  large  payloads  along  the  Space 
Station  truss  structure,  the  SSRMS  can 
transfer  power,  data  and  video  signals 
from  attached  payloads  via  the  latching 
end  effectors  at  both  ends  of  the  arm. 
These  end  effectors  allow  either  end  of 
the  SSRMS  to  receive  power  and 
operate  from  any  Power-Data  Grapple 
Fixture  on  the  Space  Station.  They  also 
permit  the  SSRMS  to  change  locations 
on  the  exterior  of  the  Space  Station  by 
“inch-worming”  from  worksite  to 
worksite. 

The  second  Canadian  system  is  the 
Special  Purpose  Dexterous 
Manipulator  (SPDM),  a  dual-arm 
dexterous  robotic  system  composed  of 
two  7-DOF  manipulators,  a  Power-Data 
Grapple  Fixture,  and  supporting 
structures  and  tooling.  The  SPDM  is 
controlled  during  teleoperations  with 
two  3-DOF  hand  controllers  and  via 
keyboard  entry  and/or  preprogrammed 
sequences  for  automated  trajectory 
control.  Each  manipulator  is 
controlled  separately,  in  addition  to 
independent  control  for  the  SPDM  body 
and  the  SSRMS  (during  operations 
where  the  SPDM  is  positioned  by  the 
SSRMS).  Human-in-the-loop  and 
automated  trajectory  control  modes  are 
supported.4 

At  the  same  time,  Japan  is  preparing  a 
dual-manipulator  system  as  an 
element  of  the  Space  Station  Japanese 


Experiment  Module  (JEM).  Composed 
of  the  Main  Arm  and  Small  Fine  Arm, 
the  JEM  Remote  Manipulator  System 
(JEMRMS)  is  intended  to  provide 
maintenance,  servicing  and  changeout 
of  science  packages  placed  on  the  JEM 
exposed  experiment  carrier.  The  Main 
Arm,  similar  in  configuration  to  the 
SRMS  and  SSRMS,  is  a  6-DOF 
positioning  tool  used  to  transport  large 
payloads  and  provide  coarse 
positioning  for  smaller,  more 
dexterous  manipulators.  The  Small 
Fine  Arm  is  a  6-DOF  manipulator 
which  can  be  operated  either  from  the 
end  of  the  Main  Arm,  or  from  a  fixture 
on  the  exposed  experiment  facility.  The 
JEMRMS  can  be  controlled  either  with 
local  teleoperation,  or  with  position- 
based  preprogrammed  motions.  The 
system  will  be  launched  on  the  same 
Space  Station  assembly  flight  as  the 
JEM  pressurized  module,  and  then 
followed  by  the  JEM  exposed 
experiment  facility  which  it  will 
support.5 

Under  development  by  Martin-Marietta 
Corporation  and  the  NASA  Johnson 
Space  Center,  the  Dexterous  Orbiter 
Servicing  System  (DOSS)  is  being 
developed  to  provide  dexterous 
manipulation  capability  for  operations 
in  the  Space  Shuttle  payload  bay.  The 
DOSS  utilizes  the  flight  manipulator 
delivered  by  the  OACT  Flight 
Telerobotic  Servicer  (FTS)  Technology 
Capture  Program  to  provide  dexterous 
manipulation  capability  for  the  STS 
Orbiter  payload  bay.  It  is  the 
culminating  element  of  a  national 
investment  of  over  $250M  in  dexterous 
space  robotics  development  that  spans 
eight  years.  The  DOSS  is  an  MPESS- 
mounted  robot  that  can  operate  from  a 
fixed  base  or  from  the  end  of  the 
Remote  Manipulator  System  (RMS). 
The  purpose  of  this  arm  is  to  provide 
the  crew  and  mission  controllers  with 
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an  alternative  to  EVA.  for  performing 
payload  bay  operations.  These 
activities  include  EVA  worksite  setup, 
payload  operations  both  nominal  and 
contingency  (e.g.  subsystem 
deployments),  and  many  Orbiter 
contingency  operations  (e.g.  repairs). 
The  DOSS  is  designed  for  repetitive 
flights  and  is  intended  to  become  a 
baseline  capability  for  STS  servicing 
operations  and  also  function  as  a 
testbed  for  future  telerobotic 
experiments.  Simultaneously,  the  first 
DOSS  flight  experiment  will  be  used  to 
validate  components  and  techniques  to 
be  used  in  the  assembly  and 
maintenance  of  the  International 
Space  Station  Alpha  (ISSA). 

The  first  flight  of  the  DOSS,  currently 
scheduled  for  June  1996,  will 
demonstrate  and  evaluate  the 
performance  of  the  dexterous  robotics 
manipulator  system  in  space  through 
the  performance  of  Space  Station-type 
maintenance  and  servicing  tasks.  This 
will  include  the  characterization  and 


verification  of  the  manipulator  system 
in  a  micro-gravity,  thermal/vacuum 
environment  and  demonstrate  Space 
Station  Technology  Flight  objectives 
while  proximal  to  Russian  MIR  space 
station  as  part  of  the  ISSA  risk 
mitigation  effort. 

DOSS  is  central  to  two  major  flight 
proposals  that  are  being  pursued  by  the 
International  Space  Station  Office:  The 
DOSS  Risk  Mitigation  Flight 
Experiment,  and  the  American  Fine 
Arm  Operational  Flight  System.  Both 
of  these  systems  are  built  around  the 
Flight  Telerobotic  System  flight 
manipulator  and  the  Hydraulic 
Manipulator  Test  Bed  that  were 
produced  as  a  result  of  the  OACT  FTS 
Technology  Capture  Program.  As  of 
April  8,  1994,  the  Space  Station  Risk 
Mitigation  Program  has  planned 
participation  in  the  DOSS 
implementation,  complimenting  the 
OACT  activity.  The  OSSI  Hubble  Space 
Telescope  Servicing  Program  may 
participate  in  the  project  starting  in  FY 
95  based  on  the 
results  of  their 


Figure  4:  Potential  configuration  for  the  Dexterous 
Orbiter  Servicing  System  flight  experiment 


ongoing 

requirements 

analysis. 

The  DOSS  flight 
experiment  will 
demonstrate 
technology  and 
techniques  which  are 
required  elements  for 
the  planned  robotic 
assembly  and 
maintenance 
operations  for  Space 
Station.  The  results 
of  the  DOSS  flight 
experiment  will 
directly  impact  the 
design,  development 
and  implementation 


of  the  system  which  will  provide 
dexterous  robotic  capabilities  for  ISSA, 
and  offers  the  only  opportunity  to  flight 
validate  these  technologies  before  the 
planned  delivery  of  the  operational 
system  (either  SPDM  or  AFA,  described 
below). 

Simultaneously,  the  OACT  T  elerobotics 
Program  is  currently  negotiating  with 
the  Space  Shuttle  program  office  to 
determine  the  long-range  plans  for 
operational  utilization  of  the  DOSS 
following  the  flight  experiment. 
Following  the  first  flight  in  CY  1996, 
the  project  will  transition  the  device 
over  to  the  STS  Operations  community, 
where  the  DOSS  will  be  integrated  in  as 
an  operational  element  of  the  STS 
servicing  toolkit.  Ongoing  operations, 
maintenance,  and  support  costs  will  be 
provided  by  STS  Operations  and  the 
user  missions.  These  negotiations  are 
not  yet  completed,  but  it  is  believed  that 
the  STS  program  office  will  be  a  full 
partner  in  the  project  once  the  current 
study  is  complete. 

The  Ranger  project  is  being  conducted 
at  the  University  of  Maryland,  under 
sponsorship  of  the  NASA  Office  of 
Advanced  Concepts  and  Technology 
Telerobotics  Program.  Over  the  past 
decade,  NASA  has  been  actively 
researching  telerobotics  technology  to 
facilitate  future  operations  in  space. 
The  Ranger  vehicle,  shown  here  in  a 
computer  generated  simulation,  is 
designed  to  be  the  first  low-cost  flight 
experiment  to  extend  these  laboratory 
investigations  to  space  flight 
experience.  Ranger  is  a  dual-arm  free 
flying  telerobotics  flight  experiment 
which  will  conduct  on-orbit  validation 
and  verification  of  many  of  the 
technologies  developed  by  the  NASA 
program.6 

To  better  understand  the  capabilities 


and  limitations  of  current  methods  for 
simulating  the  space  environment  on 
Earth,  two  Ranger  vehicles  are  being 
built.  The  first  one  is  designed  to 
operate  underwater,  and  will  be 
extensively  tested  at  the  Neutral 
Buoyancy  Research  Facility  on  the 
University  of  Maryland  campus  to  get 
basic  data  on  its’  operations  and 
capabilities.  The  second  vehicle,  as 
nearly  identical  to  the  first  as  practical, 
is  currently  scheduled  for  flight  in  late 
1996  aboard  an  expendable  launch 
vehicle.  The  project  will  correlate 
neutral  buoyancy  robotic  simulations 
by  developing  nearly  identical 
underwater  and  space  flight  units,  and 
performing  identical  tasks  in  both 
environments. 

On  orbit,  Ranger  will  demonstrate  a 
variety  of  tasks  necessary  for  future 
space  operations,  from  the  simple 
replacement  of  standard  Orbital 
Replacement  Unit  (ORU)  modules,  to 
complex  satellite  servicing  and 
refueling  tasks  which  to  date  have  only 
been  performed  by  astronauts  in  space 
suits. 

The  project  includes  collaboration  with 
the  Jet  Propulsion  Laboratory,  Langley 
Research  Center,  and  the  Michigan 
Space  Automation  and  Robotics  Center. 
It  will  carry  experiments  from  every 
NASA  center  involved  in  the  NASA 
Telerobotics  Reserach  Program,  as  well 
as  experiments  from  industry  and 
other  universities.  Utilizing 

telepresence  ground-based  control, 
coordinated  manipulator  operation, 
automated  rendezvous  and  docking 
technology,  and  a  hybrid  propulsion 
system,  Ranger  will  conduct  a 
simulated  satellite  servicing  exercise  to 
characterize  the  operational 
capabilities  of  free-flying  robotic 
systems. 


Figure  5:  The  Ranger  telerobotic  flight  experiment 
Utilizing  telepresence  ground-based  will  be  designed  as  the  first  in  what  is 


control,  coordinated  manipulator 
operation,  automated  rendezvous  and 
docking  technology,  and  a  hybrid 
propulsion  system,  Ranger  will 
conduct  a  simulated  satellite  servicing 
exercise  to  characterize  the  operational 
capabilities  of  free-flying  robotic 
systems.  Development  costs  for  the 
system  are  being  reduced  by  making 
selective  use  of  Class-S  and  Mil-Spec 
components,  maximizing  the  use  of 
commercial-quality  components, 
maximizing  in-house  operations  on  the 
flight  hardware,  compressing  the 
hardware  development  cycle  by 
concurrent  engineering  of  the  neutral 
buoyancy  and  flight  designs  and  using 
neutral  buoyancy  hardware  to  validate 
flight  systems.  Both  Ranger  vehicles 


hoped  to  be  a  long  series  of 
opportunities  to  augment  technical 
education  in  the  United  States  by 
encouraging  direct  student 
involvement  in  exciting,  important 
space  research.  Ranger  will  represent 
a  new  class  of  low-cost  expendable 
robots  designed  for  research  and 
servicing  in  areas  beyond  the  reach  of 
the  Space  Shuttle.7 

Japan  is  also  developing  a  free-flying 
robotic  servicing  experiment, 
scheduled  for  flight  in  1997  aboard  a  fi¬ 
ll  rocket.  The  National  Space 
Development  Agency  of  Japan 
(NASDA)  and  the  Ministry  of 
International  Trade  and  Industry 
(MITI)  are  developing  this  experiment 
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to  verify  automated  rendezvous  and 
docking  technologies.  A  target  vehicle 
and  chase  vehicle  will  be  deployed  to 
exercise  technologies  including  GPS 
receivers,  rendezvous  radar,  proximity 
CCD  sensors,  docking  mechanisms 
and  onboard  guidance  computers. 
Simultaneously,  a  6-DOF  manipulator 
mounted  on  the  chase  vehicle  will  be 
used  to  demonstrate  cooperative  control 
of  the  chase  vehicle  attitude  as  it  reacts 
to  manipulator  position,  ground-based 
teleoperation  of  the  manipulator, 
demonstration  of  on-orbit  satellite 
servicing  including  fuel  transfer  and 
battery  exchange,  and  target  vehicle 
acquisition,  grappling  and  restraint.8 
It  is  interesting  to  note  that  NASDA 
has  publicly  stated  they  consider  it  a 
matter  of  national  pride  that  ETS-VII 
fly  before  the  Ranger  experiment,  and 
are  currently  attempting  to  accelerate 
the  implementation  schedule  to  enable 
an  earlier  launch  date. 


Science  Payload  Servicing 

Science  payload  servicing  robotics 
differ  from  the  EVR  systems  in  that 
they  are  designed  to  maintain 
experiment  payloads  in  controlled 
environments,  and  are  specifically 
designed  as  elements  of  nominal 
experiment  operations  (ie.  the  robot  is 
intended  to  be  a  functional  component 
of  the  overall  experiment,  performing 
tasks  such  as  reagent  replenishment, 
product  harvesting,  sample  collection, 
etc.),  and  not  just  as  contingency  and 
repair  systems  in  the  event  of 
experiment  failure  or  malfunction. 
The  functions  performed  by  the 
servicer  are  intended  to  off-load  the 
requirements  for  intensive  astronaut 
maintenance  of  these  payloads,  and 
permit  operation  of  the  payloads  during 
periods  when  astronauts  may  not  be 
present. 


At  least  two  such  systems  are  currently 
in  the  final  stages  of  preflight 
integration.  McDonnell-Douglas  has 
recently  completed  development  of 
Charlotte,  named  for  the  fictional 
arachnid  star  of  Charlotte’s  Web. 
Charlotte  is  a  small  (approximately  two 
cubic  feet)  robot  which  is  physically 
connected  to  it’s  work  environment 
with  a  series  of  eight  Kevlar  strands. 
The  strands  extend  from  the  corners  of 
the  robot’s  rectangular  body  to  hard 
points  at  the  extreme  corners  of  the 
workspace,  which  may  be  the  interior 
of  SpaceLab,  SpaceHab  or  a  space 
station  module.  By  increasing  and 
releasing  tension  on  selected  strands, 
the  body  of  the  robot  is  able  to  translate 
throughout  the  entire  volume  of  the 
workspace.  Position  accuracy  and 
repeatability  is  surprising,  and  the 
device  has  no  problem  positioning  it’s 
manipulator  to  utilize  the  physical 
interfaces  associated  with  the  front- 
panel  controls  of  most  experiments. 
Charlotte  is  currently  scheduled  for 
flight  on  the  SpaceHab-3  mission. 

The  Robotic  Operated  Materials 
Processing  Systems  (ROMPS)  is  a  joint 
project  between  the  NASA  Goddard 
Space  Flight  Center,  the  Michigan 
Space  Automation  and  Robotics  Center, 
and  the  Zymark  Corporation.  The 
objective  of  ROMPS  is  to  demonstrate 
lowered  costs  of  on-orbit  processing 
through  the  use  of  robotics  to 
autonomously  produce  semi-conductor 
materials.  Scheduled  for  launch  on 
STS-64,  this  GAScan  experiment  will 
investigate  zero-gravity  annealing  of 
semi-conductor  thin  films.  Contained 
within  the  experiment  canister  will  be 
a  conventional  terrestrial  laboratory 
automation  robot  qualified  for  space 
use.  This  robot  will  utilize  low-level 
automation  to  maintain  the  materials 
furnace,  supply  source  substrates  to 
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the  furnace  and  harvest  processed  thin 
films.  Specific  issues  overcome  during 
the  qualification  of  the  terrestrial  robot 
for  use  in  the  on-orbit  materials 
processing  facility  include  exclusion  of 
hydrocarbon-based  lubricants,  use  of 
low  off-gassing  materials, 
incorporation  of  low  particulate- 
producing  mechanisms,  and 
integration  of  force  sensing  for  reliable 
handling  of  fragile  films. 

It  is  also  appropriate  to  mention  that 
the  European  Space  Agency  is 
investigating  the  incorporation  of  a 
large-scale  science  payload 
maintenance  robot  system  into  the 
Columbus  module  of  Space  Station. 
This  system  would  have  a  work 
envelope  encompassing  the  entire 
interior  of  the  module,  and  would 
provide  logistics  support  for  minimally- 
tended  science  experiments  and 
materials  production  systems. 

Planetary  Surface  Systems 

Of  the  three  classes  of  space  robotic 
systems,  planetary  surface  robotics  is 
one  in  which  is  largest  breadth  of 
knowledge  exists,  although  it  is 
somewhat  dated  knowledge.  As  early 
as  1967,  the  Surveyor  missions  carried 
simple  r  e  m  o  t  e  1  y  -  o  p  e  r  a  t  e  d 
manipulators  to  the  surface  of  the 
Moon  to  collect  samples  of  the  Lunar 
regolith.  Followed  by  the  Russian 
Lunakhods  in  1969  and  1980,  and  the 
Viking  missions  to  Mars  in  1976,  these 
early  efforts  identified  the  fundamental 
environmental  constraints  and 
technology  obstacles  to  be  surmounted 
to  enable  the  development  of  robust, 
long-lived  planetary  surface  robotics. 

It  was  traditionally  accepted  that  the 
next  generation  of  robotic  rovers  for 
unmanned  Lunar  and  Mars  missions 


would  be  large  (800-Kg  or  more), 
monolithic,  highly  intelligent  and 
autonomous  devices  which  would 
require  significant  development  and 
operational  support  in  terms  of 
technology,  budget,  computational  and 
human  resources.  Then  in  1989,  a 
small  group  of  rogue  technologists  at 
MIT  and  JPL  began  an  new  initiative 
in  micro-rover  technology  based  on 
subsumption  architectures.  Making 
use  of  progressively  smaller 
computers,  increasingly  advanced 
sensors,  and  maturing  mobility 
systems,  a  series  of  micro-rover 
testbeds  was  developed  which 
culminated  in  the  MESUR  (Mars 
Environmental  Survey)  Pathfinder 
Rover.  This  six-wheeled  5  Kg-class 
rover  is  scheduled  for  launch  to  Mars 
in  1996,  and  will  perform  technology 
validation  experiments  in  addition  to 
science  investigations  and  instrument 
deployment.  Control  for  the  rover  will 
be  shared  between  Earth  and  the 
limited  onboard  intelligence  of  the 
rover.  The  Earth-bound  operator  will 
use  camera  views  from  the  MESUR 
lander  to  designate  major  terrain 
features  as  navigation  waypoints  in  the 
vicinity  of  the  lander  to  be  either 
avoided  or  visited  by  the  rover.  By 
combining  sensory  input  with 
predefined  “behaviors”  the  rover  will 
autonomously  navigate  between  the 
waypoints,  avoiding  rocks,  crevasses 
and  other  impassable  terrain.  At  the 
same  time,  the  minimal  onboard 
computational  system  will  be  working 
overtime  to  store,  compress,  packetize 
and  transmit  rover  imagery  and  other 
telemetry  back  to  the  lander  via  UHF 
modem,  where  it  is  then  relayed  to 
Earth.  To  maintain  an  extremely  low- 
cost  funding  profile  (as  compared  with 
previous  planned  systems  such  as  the 
Mars  Rover  Sample  Return  project), 
the  rover  design  philosophy  duplicates 
that  of  the  Ranger  project:  extensive 
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use  of  commercial-  and  military- 
quality  components,  maintaining  a 
small  development  team,  rapid 
prototyping  with  concurrent 
engineering,  streamlining  of 
component  procurement  and 
qualification  procedures,  and  intense 
industry  involvement  and  buy-in. 

Also  scheduled  for  flight  in  1996, 
Russia  intends  to  launch  the  Mars  ‘96 
mission  which  will  include  the 
Marsokhod  rover.  Being  developed  by 
the  Institute  for  Space  Research  (IKI) 
and  the  Babakin  Center  of  NPO 
Lavochkin,  the  Marsokhod  is  a  six¬ 
wheeled,  100-Kg  rover  will  use 
radioisotopic  thermal  generators  (RTG) 
for  power  generation  and  thermal 
control.  Because  of  the  RTGs,  the  rover 
will  be  able  to  operate  during  the 
Martian  night,  and  is  expected  to  have 
a  long  surface  lifetime  (one  year  or 
more)  with  a  potential  total  excursion 
distance  of  over  100  kilometers.  The 
Marsokhod  enables  exceptional 
mobility  characteristics  through  the 
use  of  unique  bi-conic  titanium  wheels 


and  a  segmented  three-part  chassis. 
IKI  is  currently  discussing  possible 
United  States  participation  in  the 
project,  which  would  incorporate 
telepresence  control  technology 
developed  by  the  NASA  Ames  Research 
Center  and  a  lightweight  6-DOF 
manipulator  system  developed  by 
McDonnell-Douglas  into  the  flight 
system.  Field  tests  of  the  Ames  Virtual 
Environment  Vehicle  Interface  (VEVI) 
have  already  been  conducted  with  a 
prototype  Marsokhod  rover  located  on 
the  Russian  Kamchatka  Peninsula, 
and  the  operators  located  in 
Huntington  Beach,  California.9 

In  addition  to  these  Mars-bound  rovers, 
LunaCorp,  a  Virginia-based 
corporation,  has  recently  announced 
plans  for  a  lunar  rover  project  slated 
for  launch  in  1998.  Rather  than  driven 
by  science  needs,  the  incentive  for  this 
project  is  primarily  entertainment  - 
the  goal  of  the  project  is  to  provide  the 
world's  first  interactive  space 
exploration  event  by  giving  the  public 
the  opportunity  to  drive  the  rover  on  the 
moon.  The  rover  will  be 


Figure  6:  A  prototype  of  the  Russian  Marsokhod  rover 
undergoes  terrestrial  field  tests  in  Death  Valley 


remotely  operated  via 
telepresence  control  from 
workstations  located  in  theme 
parks  around  the  country. 
Funding  for  the  project  is  to 
come  from  theme  park 
operators,  television  networks, 
corporate  sponsors  and 
advertising  agencies. 
Capitalizing  on  NASA  rover 
technology  developments, 
LunaCorp  is  working  with 
Carnegie-Mellon  University  to 
transfer  these  technologies  into 
the  first  commercial  lunar 
rover  application.  The  project 
will  use  a  Russian  Phobos 
launch  system,  purchased 
through  International  Space 
Enterprises,  to  deliver  the 
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lander  and  rover  to  lunar  orbit. 
Following  descent  and  landing,  the 
rover  will  initially  explore  in  the 
vicinity  of  the  Apollo  11  landing  site, 
and  then  initiate  a  traverse  across  the 
Lunar  surface  of  up  to  1000  kilometers. 
A  combination  of  predictive  display, 
telepresence  and  virtual  reality  control 
systems  will  be  used  to  compensate  for 
the  4-8  second  time  delay  associated 
with  communicating  across  the  Earth- 
Moon  distances.  Detailed  virtual 
reality  reconstructions  of  the  regions 
explored  by  the  rover  will  allow 
potential  rover  drivers  to  practice 
navigating  the  rover  in  simulation 
before  they  take  over  control  of  the  real 
system.  While  most  of  the  technology 
obstacles  associated  with  this  concept 
have  already  been  addressed,  the  real 
hurdle  will  be  the  creation  of 
appropriate  system  intelligence  which 
can  react  in  real  time  to  prevent  an 
operator  from  driving  the  rover  into  a 
system-fatal  situation. 

Terrestrial  T  echnology  T  ransfer 

While  the  space  applications 
development  efforts  continue  as  full 
pace,  parallel  efforts  are  being  driven  to 
force  the  new  technologies  developed  by 
these  efforts  into  terrestrial 
applicability.  As  each  on-orbit  robotic 
system  moves  closer  to  actual  flight,  by¬ 
products  of  the  effort,  such  as 
fundamentally  new  robotic  joint 
designs,  exoskeleton  systems, 
fundamental  robotic  control  theory 
development,  and  widely-applicable 
proximity  sensor  technology  are 
produced.  Each  of  these  component 
technologies  hold  the  promise  of 
improving  terrestrial  robotic  systems 
for  both  space-  and  non-space  related 
uses.  The  NASA  robotics  program 
provides  a  mechanism  for  the 
application  of  developed  technologies 


into  terrestrial  task  environments. 
These  tasks  move  the  technologies 
developed  in  the  other  elements  of  the 
program  from  the  laboratory  setting 
into  operational  use,  and  take 
advantage  of  the  relatively  easy 
terrestrial  access,  well  understood 
environments,  and  myriad  problems  to 
be  solved  to  demonstrate  the 
applicability  of  space  telerobotics. 

Throughout  the  life  of  the  NASA 
Telerobotics  Program,  NASA  has 
worked  to  build  and  maintain 
coordination  with  other  government 
robotics  programs,  including  those  of 
the  National  Science  Foundation  (NSF) 
and  the  National  Institute  of  Standards 
and  Technology  (NIST).  These  efforts 
include  cooperative  activities, 
collaborative  research,  and  external 
transfer  of  NASA-developed  robotics 
technology.  These  efforts  have  three 
purposes:  to  develop  industrial 

applications  of  telerobotics  technology, 
to  apply  telerobotics  technology  to 
terrestrial  science  and  research 
efforts, and  to  strengthen  intra¬ 
government  coordination.  Several  of 
the  activities  are  summarized  below, 
beginning  with  the  efforts  targeting 
development  of  industrial  applications 
of  telerobotics  technology:10 

•  The  Automated 
Manufacturing  Research 
Program,  conducted  by  NIST 
is  investigating  automation  in 
factory- floor  settings,  and  the 
relative  advantages  of 
improved  work  cells  against 
more  capable  manipulation 
systems.  NASA  participates 
in  the  annual  program  review 
conducted  by  NIST,  and 
coordinates  with  NIST  to 
transfer  NASA-developed 
robotics  workcell  technology 
into  this  effort. 


In  previous  years,  NIST  and 
the  NASA  telerobotics  efforts 
have  cooperatively  developed 
several  new  technologies  and 
architectures  for  the  control  of 
robotic  systems.  For  example, 
the  NASREM  robot  control 
architecture  was  jointly 
developed  by  NASA  and  NIST 
as  a  precursor  to  the  NASA 
Flight  Telerobotic  Servicer 
program.  The  architecture  is 
now  used  as  a  standard 
architecture  definition 
methodology  by  many  NASA, 
NIST  and  industry  projects. 
NASA  has  directly  supported 
NIST  in  several  of  these 
cooperative  activities,  with 
annual  funding  for  robotics 
research  reaching  up  to  $1 
million  per  year. 

The  NASA  and  NSF  robotics 
research  programs  have 
jointly  co-sponsored  the 
“Bilateral  Exchanges  on  the 
Approaches  to  Robotics  in  the 
United  States  and  Japan” 
conference,  which  conducted 
investigations  into  the 
methods,  techniques  and 
technologies  used  by 
government  and  industry  to 
research  and  develop 
fundamental  new  robotics 
technologies.  The  outcome  of 
this  activity  was  publication  of 
a  manuscript  which 
contrasted  the  approaches 
used  in  the  United  States  and 
Japan,  and  which  offered 
NASA  and  NSF  insights  into 
the  content  of  the  robotics 
development  programs 
supported  by  MITI,  NASDA, 
and  several  Japanese 
industries. 


•  The  Advanced  Research 
Projects  Agency  (ARPA)  has 
selected  the  Langley  Research 
Center  robotics  program  as 
one  of  their  technical  agents 
in  the  area  of  robotics.  Under 
this  agreement,  LaRC  and 
DARPA  cooperatively  issue 
university  research  grants  to 
sponsor  the  development  of 
innovative  new  robotics 
technologies,  as  well  as 
increase  robotics  educational 
expertise  in  the  United  States. 

•  The  program  has  maintained 
close  ties  with  the  U.  S.  space 
robotics  industrial 
community,  and  monitored 
industrial  developments  of 
potential  applicability  to  the 
NASA  space  robotics  and 
planetary  rover  research 
efforts.  For  example,  the 
Martin-Marietta  Corporation 
participates  in  the  Telerobotics 
Intercenter  Working  Group, 
and  in  technical  program 
reviews  and  assessments 
such  as  the  Space  Systems 
Technology  Advisory  Council. 
This  coordination  facilitates 
the  transfer  of  NASA- 
developed  technologies  to  the 
space  robotics  industry,  and 
aids  in  the  rapid  application  of 
these  technologies  to 
terrestrial  manufacturing 
and  automation  problems. 

•  The  program  coordinates  with 
several  robotics  industry 
advisory  and  technology 
interchange  groups,  to 
facilitate  the  transfer  of 
NASA-developed  technology  to 
the  industrial  community  and 
receive  comments  on  the 
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overall  direction  and  focus  of 
the  program.  One  such  group 
is  the  Space  Automation  and 
Robotics  Technical  Committee 
(SARTC)  of  the  American 
Institute  of  Aeronautics  and 
Astronautics  which  meets 
three  or  four  times  annually 
with  the  charter  of 
disseminating  information 
about  space  automation  and 
robotics  and  promoting  the 
technology  to  industry, 
academia,  and  government. 
The  SARTC  is  composed  of 
industry  representatives  from 
the  aerospace  community,  as 
well  as  government  and 
academia. 


Some  of  the  efforts  which  target 
application  of  telerobotics  technology 
to  terrestrial  science  and  industry 
efforts  are  listed  below: 

•  Several  programs  sponsored 
by  NSF  both  sponsor  and 
utilize  telerobotics  and  robotics 
technology  research  and 
development.  In  1992  NASA 
and  NSF  cooperated  in 
conducting  the  _Mt.  Erebus 
Explorer  project,  a  project  to 
deploy  a  robot  into  the  interior 
of  a  volcano  crater  in  the 
Antarctic.  This  project, 
conducted  as  part  of  the 
Telerobotics  Program  and  the 
Antarctic  Space  Analog 
Program,  demonstrated 
innovative  new  robotics 
technologies  developed  by 
NASA.  It  is  anticipated  that 
this  project  will  spawn  several 
new  activities  which  may 
revolutionize  volcanic  sample 
collection  and  lead  to 
significant  new  applications  of 
robotics  in  terrestrial  field 


science  operations.  This 
project  is  being  continued  with 
the  United  States  Geological 
Survey,  and  will  deploy  the 
Dante  robot  to  a  volcano  in 
Alaska  in  the  summer  of  1994. 

•  In  addition  to  the  involvement 
with  the  NSF  Polar  Programs 
Division  (which  cooperated 
with  the  Mt.  Erebus  Explorer 
project),  NASA  is  currently 
negotiating  with  the  NSF 
Oceans  Division  to  investigate 
the  potential  for  application  of 
NASA-developed  robotics 
technology  to  underwater 
science  sampling  operations. 
Of  particular  interest  is  the 
underwater  Remotely 
Operated  Vehicle  (ROV) 
technology  which  NASA 
developed  and  demonstrated 
under  the  Antarctic  sea  ice 
with  the  cooperation  of  NSF  in 
1992.  Additional  negotiations 
are  underway  with  the  NSF 
Information,  Robotics  and 
Intelligent  Systems  Division  to 
jointly  sponsor  robotics 
research  and  investigate 
opportunities  for  transfer  of 
NASA-developed  robotics 
technologies  to  NSF  grantees 
and  research  programs. 

•  The  robotics  laboratories  at  the 
Jet  Propulsion  Laboratory 
have  been  working  with 
Computer  Motion,  Inc.  to 
develop  technologies  for 
applications  where  human 
ability  to  perform  a  task  is 
limited  by  human  dexterity 
and  physical  capabilities.  One 
specific  application  has  been 
in  minimally  invasive 
laproscopic  surgery.  This  type 
of  medical  procedure  makes 


use  of  remote  cameras,  known 
as  laproscopes,  which  are 
typically  held  by  an  assistant 
to  the  surgeon  during  a 
procedure.  The  assistant  has 
control  of  the  surgeons  field  of 
view,  and  the  surgeons 
performance  is  often  limited 
by  the  efficiency  of 
communication  with  the 
assistant.  To  address  this 
problem,  the  project  has 
developed  the  Automated 
Endoscopic  System  for 
Optimal  Positioning  (AESOP), 
a  robotic  assistant  which 
holds  the  laproscope  and  is 
guided  by  the  surgeon  with  a 
foot-  and/or  hand-controlled 
interface.  Thus  the  surgeon  is 
able  to  gain  control  of  the 
viewfield  by  direct 
coordination  between  himself 
and  a  robotic  assistant.11 

JPL  has  also  worked  with 
Cybernet  Systems 
Corporation  to  develop  the 
PER-Force  hand  controller 
which  manipulates  robots  or 
objects  by  “feel”.  this  small 
backdrivable  robot  is 

combined  with  advanced 
machine  vision  processing 
and  enhanced  computer 
generated  visual/tactile  force 
feedback  cues  to  enable  an 
enhanced  interface  for  the  use 
on  hazardous  environment 
operations.  This  system  has 
been  implemented  with  a  goal 
of  integrating  it  within  the 
manufacturing  environment 
and  tasks  which  have  no 
immediate  solution  with  hard 
automation  or  changes  in 
methodology  or  workcell 
design.  One  example 

application  being  developed  is 


pick-and-place  operations  for 
automobile  transmission 
packing.12 

•  As  an  offshoot  of  work 
sponsored  by  the  program,  the 
Stanford  University  Aerospace 
Robotics  Laboratory  and  Real- 
Time  Innovations,  Inc.  have 
developed  ControlShell,  a  next 
generation  CASE  framework 
for  real-time  system  software 
development.  ControlShell 
includes  many  system¬ 
building  tools,  including  a 
graphical  flow  editor,  a 
component  data  requirement 
editor,  a  state-machine  editor, 
a  distributed  data  flow 
manager,  an  execution 
configuration  manager,  an 
object  database  and  a  dynamic 
binding  facility.  ControlShell 
is  being  used  in  several 
applications,  including  the 
control  of  free-flying  robots, 
underwater  autonomous 
vehicles,  and  cooperating-arm 
robotic  systems.13 

•  NASA  has  teamed  up  with 
Limbs  of  Love  and  a  group  of 
medical  and  prosthetics 
specialists,  prosthetics  users, 
insurance  industry 
representatives,  and 
university  researchers  to 
identify  research  objectives  in 
prosthetic  limbs.  As  part  of 
this  effort,  the  NASA  Johnson 
Space  Center  has  been  actively 
working  with  Rice  University 
to  improve  dexterous  hand 
design  and  to  develop  a 
method  for  myoelectric  control 
of  multifinger  hands.  In 
theory,  myoelectric  control  of 
robotic  hands  will  require 
little  or  no  mechanical  parts 
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and  will  greatly  reduce  the 
bulk  and  weight  usually  found 
in  dexterous  robotic  hand 
control  devices.  An 

improvement  in  myoelectric 
control  of  multifinger  hands 
will  also  benefit  prosthetics 
users.14 

This  list  is  not  exhaustive,  but  is  a 
representative  cross-section  of  the  type 
of  activities  conducted  by  the  NASA 
Telerobotics  Program  and  other 
government  organizations.  Additional 
efforts  have  extended  this  coordination 
to  industrial  telerobotics  research 
programs,  to  aid  in  the  transfer  of 
government-developed  technology  to  the 
U.S.  commercial/industrial  robotics 
community.  These  efforts  use  two 
mechanisms  to  transfer  the  technology 
developed  by  the  program. 

The  first  mechanism  pairs  NASA 
researchers  and  commercial 
developers  together  to  develop  space 
telerobotics  technology  which  is  based 
on  commercially-available  products. 
As  the  terrestrial  systems  are  extended 
to  address  the  needs  of  the  space 
telerobotics  program  by  the 
researchers,  the  commercial  partners 
are  able  to  identify  markets  and 
applications  for  dual-use 
implementations  of  the  new 
technologies,  and  rapidly  incorporate 
them  into  new  product  lines.  An 
example  of  this  is  the  development  of 
the  “phantom  robotic  control” 
technology  developed  by  JPL  under  the 
Advanced  Teleoperation  project.  This 
technology  has  been  developed  as  an 
extension  to  the  commercial 
Interactive  Graphics  Robot 
Instructional  Program  (IGRIP) 
software  package  from  Deneb  Robotics. 
JPL  has  worked  with  Deneb  to 
smoothly  integrate  the  extension  into 
the  IGRIP  package,  and  negotiated  a 


mechanism  to  provide  this  extension  to 
Deneb  for  commercialization.  Deneb 
has  identified  a  new  need  for  this 
technology,  beyond  the  original 
application  of  space  telerobotics,  and 
plans  to  incorporate  the  extension  into 
their  commercial  product  line. 

The  second  mechanism  pairs  NASA 
researchers  with  commercial 
developers  to  work  jointly  on  the 
application  of  space  telerobotics 
technologies  to  terrestrial  problems. 
The  commercial  partners  bring 
existing  terrestrial  robotics  systems 
and  capabilities  into  the  project,  and 
work  jointly  with  NASA  researchers  to 
improve  these  systems  through  the 
application  of  space  telerobotics 
technology.  An  example  of  this  is  the 
Hazardous  Materials  Handling  Robot 
(HAZBOT)  project  at  JPL,  which  is 
being  conducted  with  the  partnership 
of  Remotec,  Inc,  and  which  is 
addressing  the  problem  of  hazardous 
chemical  spill  incident  identification 
and  mitigation  through  the  use  of 
robotics.  JPL  and  Remotec  worked  to 
apply  technologies  developed  by  the 
Telerobotics  Program  to  improve  the 
off-the-shelf  Remotec  “Andros”  mobile 
robot  to  satisfy  the  unique  needs  of  the 
HAZBOT  project.  Several  of  the  specific 
techniques  and  mechanisms  developed 
during  this  process  have  been  delivered 
back  to  Remotec  for  incorporation 
within  their  commercial  product  line.15 

The  program  has  similar  interactions 
with  other  members  of  the  U.S. 
industrial  robotics  community,  such  as 
Robotics  Research  and  Oceaneering. 
The  current  program  plans  include 
expanding  these  efforts  to  include  a 
larger  percentage  of  the  U.S.  robotics 
industry. 
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Technology  Requirements  for  Future 
Systems 

With  the  advent  of  these  new 
experimental  and  operational  space 
robotic  systems,  the  ability  for  remote 
manipulation  to  perform  real  tasks 
which  offer  significant  improvements 
to  mission  operations,  cost  effectiveness 
and  mission  safety  will  be  proven.  But 
these  will  still  be  early  generations  of 
advanced  space  robotic  applications. 
As  successive  waves  of  space  robotic 
applications  are  deployed  beyond  the 
year  2000,  the  goal  of  intelligent, 
autonomous  space  robotics  will  become 
more  and  more  important.  Technology 
drivers  for  these  systems  include 
enhanced  collision  detection  and 
avoidance,  advanced  local  proximity 
sensing,  task  level  control 
workstations,  improved  command  and 
control  architectures,  fault  tolerant 
architectures,  reduced  mass  and 
volume,  worksite  recognition  and 
representation,  improved  robotic 
dexterity,  advanced  supervisory 
control,  and  improved  overall  system 
robustness. 

By  combining  these  next-generation 
technologies  with  the  operational 
knowledge  gained  from  applications 
being  flown  in  the  next  few  years,  the 
first  intelligent  space  robotic  systems 
will  be  within  reach.  By  then 
combining  the  technologies  with  the 
development  procedures  utilized  by  the 
current  suite  of  applications,  the  next 
generation  of  space  robotic  applications 
will  be  affordable,  even  within  the  ever- 
tightening  budget  environment  of 
today’s  space  program. 
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SUMMARY 

A  robotics  system  is  composed  of  a  hardware  platform  which  provides  the  stability,  mobility  and 
manual  dexterity;  and  a  software  system  which  acts  to  interpret  the  input  sensor  data  and  the 
mission  statement  and  then  to  create  the  required  excitation  to  the  hardware.  Stated  in  this  rather 
simplistic  way,  there  seems  to  be  little  reason  for  the  vast  array  of  laboratories  feverously 
working  on  the  creation  of  fully  autonomous  robots.  The  overall  task  and  its  components, 
however,  has  proven  of  intricate  complexity,  with  succeeding  layers  of  hidden  problems 
becoming  exposed  just  as  a  solution  appears  eminent. 

This  paper  is  concerned  only  with  the  software  components  of  a  robotics  system,  and  in 
particular  with  that  portion  of  the  system  which  comes  under  the  general  description  of 
intelligent  software.  It  is  a  'stand-off  approach  derived  from  a  software  engineer's  point  of  view. 
The  intent  is  to  review,  from  a  particular  and  perhaps  novel  point  of  view,  the  various  software 
paradigms  and  their  intrinsic  capabilities  and  functionalities.  This  review  provides  the  bases  for 
a  detailed  map  of  the  current  capabilities  of  software  paradigms  onto  the  guidance  and  control 
problems  of  intelligent  robotics  systems. 

The  paper  is  partitioned  into  two  parts:  Machine  Intelligence  as  a  Robotics  Component,  and 
Expectations  and  Projections.  In  the  first,  a  perspective  of  intelligent  software  is  presented  which 
provides  the  bases  for  considering  guidance  and  control  problems.  In  the  second  part,  the  utility 
of  this  perspective  is  demonstrated  by  exploring  a  generic  guidance  and  control  architecture.  The 
paper  ends  by  suggesting  some  approaches  for  future  directions  in  developments  in  this  field. 


PART  A  -  INTELLIGENCE  AS  A  ROBOTICS  COMPONENT 


1.0  INTRODUCTION 

The  concept  of  a  fully  autonomous  intelligent  robot  has  been  a  reality  for  science  fiction  writers 
for  eons.  Writers  such  as  Asimov  have  even  postulated  rules  of  behavior  for  such  devices.  More 
recently,  Star  Trek  (an  American  TV  serial)  has  as  a  principal  character  'Mr.  Data,'  a  highly 
mobile  and  highly  intelligent  Android.  Given  the  spectacular  advances  in  computing  through-put 
available  on  small  low-power  silicon  chips,  and  the  equally  incredible  advances  in  the  storage 
density  of  memory  technology,  it  seems  reasonable  to  suggest  that  sufficient  computing  power 
and  memory  can  now  be  loaded  onto  a  mobile  robot  to  fulfill  almost  any  conceivable  computing 
requirement.  Indeed,  there  seems  little  technological  reasons  for  the  delay  in  creating  such  a 
device.  Certainly,  from  a  software  engineer's  point  of  view,  the  underlying  computational 
capability  is  now  widely  available  and  a  naive  assertion  is  -  "It  should  not  take  long  for  the 
implementation  of  a  really  intelligent  robot?" 

The  facts  are,  of  course,  that  the  simplest  of  human  activities  are  proving  of  inestimable 
difficulty  in  implementation;  in  most  cases  because  we  do  not  really  know  how  it  is  that  the  brain 
accomplishes  its  basic  functions.  It  is  possible,  however,  to  conjecture  at  a  high  level  what  kind 
of  processing  accomplishes  our  rudimentary  and  advanced  capabilities,  and  from  these 
conjectures  postulate  what  kind  of  software  paradigms  will  be  needed  to  emulate  these  functions. 
From  this  approach  a  general  frame  work  can  be  established  which  categorizes  the  various 
problems  and  offers  a  coherent  view  of  the  activities  around  the  world,  the  key  problems  that 
must  be  surmounted  and  the  potential  for  near-term  advancement. 
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Before  proceeding,  it  is  important  to  share  a  perspective  of  the  complexity  of  any  given  problem 
and  to  recognize  the  basic  difficulties  in  its  solution.  These  come  in  four  categories: 

i)  Unknown,  and  Perhaps  Unknowable:  In  this  category,  we  place  capabilities  that 
are,  as  yet,  not  completely  understood,  such  as  vision,  speech  understanding, 
synergistic  insights,  etc;  all  of  which,  in  the  end,  may  not  be  understandable  or 
completely  emulatable.  Such  problems  may  require  whole  new  thought 
paradigms  to  define  the  direction  of  research; 

ii)  Unknown,  and  Hopefully  Knowable:  In  this  category  are  capabilities  that  are  not 
yet  completely  understood,  but  which  are  thought  to  be  achievable  with  more 
research.  There  is  a  gray  area  between  this  category  and  the  first.  Speech 
recognition,  for  example,  may  belong  to  the  first,  while  we  hope  it  belongs  to  the 
second; 

iii)  Known,  but  Technology  Limited:  In  this  category  are  capabilities  that  will 
eventually  be  solved  by  the  advances  of  computing  technology.  Such  problems 
may  be  solvable  given  computer  power  not  yet  available,  but  foreseeable;  and, 

iv)  Known  and  Implementable:  In  this  category  tire  all  those  things  that  we  are  now 
able  to  emulate. 

It  seems  important  to  perceive  and  appreciate  these  distinctions.  Some  of  our  human 
characteristics  may  not  be  understandable,  let  alone  emulatable.  The  problem  of  speech 
comprehension  being  a  typical  example  which  has  been  'near'  solution  for  over  a  decade. 
Complete  emulation  of  the  human  vision  system,  as  a  further  example,  may  not  be  emulatable 
because  we  may  never  understand  how  we  do  it  ourselves. 

Before  addressing  the  intelligence  question,  it  is  also  acknowledged  that  some  reasonably 
difficult  problems  remain  in  other  areas  of  autonomous  systems,  such  as  duplication  of  the 
human  stability  and  mobility  capabilities.  We  leave  these  problems  to  others,  however  it  may  be 
worth  considering  where  these  fit  in  the  problem  spectrum. 

This  paper  is  presented  in  two  parts,  roughly  divided  into  a  consideration  of  intelligence  as  a 
robotics  component,  and  a  final  view  of  our  expectations  and  projections.  First,  we  propose  a 
general  logical  model  of  the  robotics  problems  as  seen  from  a  central  intelligence  perspective. 
Finally,  some  general  conclusions  are  drawn.  In  Part  B,  the  first  step  is  to  review  the  capabilities 
of  various  processing  paradigms,  and  then  to  speculate  how  these  may  work  in  concert  to  solve 
robotics  problems.  The  next  step  will  pull  all  this  together  and  suggest  viable  paths  into  the 
future  for  the  advancement  of  guidance  and  control  capabilities.  The  problem  of  data  preparation 
(sensor  fusion  and  integration)  is  considered  as  an  example.  It  is  proposed  that  the  goal  of  this 
activity  should  be  the  appropriate  preparation  of  data  for  presentation  to  the  various  intelligent 
systems. 

It  seems  important  that  the  model  of  the  problem  be  appropriate,  or  at  least  shared,  to  provide  a 
common  basis  for  discussion  or  criticism.  We  will  attempt  this  first. 
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2.0  GENERAL  LOGICAL  MODEL 

Each  of  the  disciplines  in  the  field  of  robotics  has  a  model  which  characterizes  their  view  of  their 
problem.  From  the  point  of  view  of  central  intelligence,  all  other  aspects  of  the  robot  are  simply 
a  means  of  providing  the  appropriate  data  input  from  a  bank  of  sensors,  and  the  requirement  is  to 
create  the  correct  data  output  to  a  bank  of  actuators. 

Figure  1  presents  a  simple  and  very  general  view  of  the  various  logical  levels  of  interest.  Control 
of  the  input  data  stream  may  be  centralized  or  distributed.  In  the  former,  the  central  intelligence 
decides  the  precise  command  structure  to  control  modifications,  extensions  or  other 
modifications  to  the  input  data  stream.  In  the  later,  only  high  level  requirements  are  passed  on  to 
the  next  component. 

A  version  of  the  physical  world  is  recorded  by  the  sensors,  and  is  acted  upon  by  the  actuators.  In 
between,  the  sensor  data  is  preprocessed  (possibly  fused  and/or  integrated)  and  a  state  vector 
presented  to  the  central  intelligence.  The  central  intelligence  must  acknowledge  the  state  vector, 
interpret  it  in  terms  of  its  world  model,  derive  an  appropriate  response  in  terms  of  its  allocated 
tasks,  and  create  an  output  vector  which  is  delivered  to  the  actuators. 

Figure  2  shows  a  more  complete  division  of  functions  of  the  central  intelligence,  which 
corresponds  to  the  hemispheric  division  of  functions  in  the  human  brain.  In  this  model,  the 
processing  functions  are  partitioned  into  three  parts.  The  first  part  responds  to  the  state  vector  in 
terms  of  automatic  and  predictable  processing  algorithms.  The  second  is  a  more  intuitive  part, 
which  reacts  to  the  world  in  terms  of  a  learned  response  based  on  successful  responses  to 
situations  experienced  during  its  lifetime.  The  third  could  be  termed  the  reasoning  part,  which 
assesses  the  state  vector  of  the  world  in  terms  of  logical  reasoning  based  on  experience  and  rules 
of  behaviour. 

These  three  divisions  in  capability  are  intuitively  appealing.  The  reactive  component  can  be 
implemented  by  algorithms  on  high  speed  computing  devices,  and  could  be  used,  for  example,  to 
maintain  verdcality  in  a  robot.  The  rational  part  would  resolve  problems  by  either  inductive  or 
deductive  reasoning,  based  on  the  known  interactions  of  the  attributes  of  the  world  model.  The 
intuitive  part  could  respond  to  situations  which  are  not  quantifiable  or  describable  in  a  manner 
suitable  for  implementation  in  an  algorithm,  or  by  a  rational  logical  sequence  of  deductive  or 
inductive  reasoning. 
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3.0  INTELLIGENT  PARADIGMS 

3.1  The  AI  World 

Artificial  Intelligence  (AI)  attempts  to  model  decision  problems  requiring  human-like  reasoning 
abilities.  The  field  of  AI,  as  shown  in  Figure  3,  covers  many  technologies  such  as  machine 
learning,  robotics,  natural  language  processing,  vision,  speech  recognition,  and  expert  systems. 
Some  authors,  however,  categorize  machine  learning  as  separate  from  artificial  intelligence  [1]. 
Each  AI  technology  typically  is  explained  in  terms  of  the  reasoning  and  representation 
approaches.  Representation  involves  the  form  in  which  knowledge  is  encoded,  while  reasoning 
involves  utilizing  that  knowledge  to  interpret  and  react  to  stimulation. 


3.2  A  World  Model  and  State  Vectors 

It  is  very  important  to  recognize  that  humans  and  robots  perceive  the  world  through  a  set  of 
sensors,  augmented  by  a  model  of  the  world  that  we,  as  humans,  derive  from  sociological 
conditioning  and  experience.  As  a  species  we  have  been  deprived  by  evolution  to  a  subset  of 
capabilities  enjoyed  by  other  species;  consider  the  enhanced  capabilities  of  smell  (enjoyed  by  a 
dog),  hearing  (by  most  other  animals),  and  sight.  Our  vision  system  has  neither  the  sensitivity  of 
the  cat  nor  the  range  of  an  eagle.  Indeed  the  spectrum  of  light  we  detect  is  limited  (no  infrared 
for  example).  Despite  these  severe  limitations,  we  have  developed  a  common  world-model,  and 
in  most  cases  have  derived,  through  intuition,  training  and  reasoning,  processing  procedures  to 
respond  to  our  limited  sensor  inputs.  Of  course,  to  a  limited  extent,  we  are  able  to  fuse  and/or 
integrate  data  (vision,  hearing,  smell  and  touch)  to  enhance  our  survival  skills. 

The  input  to  any  (artificial)  processing  capability  is  a  data  vector  representing  the  perceived  state 
of  the  world  obtained  from  sensors.  This  vector  can  contain  strictly  numeric  entries  or 
descriptive  attributes  of  the  world  state  or  possibly  both,  depending  on  the  requirements  of  the 
processing.  In  terms  of  the  model  developed  earlier,  there  are  several  types  of  processing 
approaches  to  derive  the  appropriate  response.  We  will  consider  several  in  turn. 


3.3  Algorithms  -  A  Reactive  Response 

An  algorithm  is  a  detailed  description  of  the  data  transformations  resulting  in  a  predictable  and 
verifiable  outcome  based  on  a  given  input  vector.  Algorithms  are,  perhaps,  the  easiest  software 
paradigm  to  program,  test  and  -  usually  -  offer  the  fastest  response  time  on  a  given  computing 
platform.  Algorithms  executing  numerical  data  have  the  additional  assistance  of  high- 
performance  arithmetical  units  and  proven  techniques  for  programming  and  data  manipulation. 

Algorithms  are  normally  not  included  in  the  class  of  intelligent  software  paradigms.  Yet 
algorithms  can  provide  highly  intelligent  and  appropriate  responses  to  input  situations.  Indeed 
the  processing  capability  of  algorithms  can  be  very  complex  and  include  profound  assessments 
of  the  state  of  the  world  and  the  range  of  appropriate  responses.  We  will  develop  the  point  of 
view  that  intelligent  software  can  include  any  paradigm,  providing  it  has  certain  attributes  to  be 
defined  later. 
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3.4  Fuzzy  Logic 

Fuzzy  logic  devices  can  accomplish  some  very  profound  responses  to  the  input  state.  Fuzzy 
logic  has,  at  its  basics,  a  means  of  ascribing  membership  in  a  set  of  classes  of  various  attributes 
of  the  world.  Thus  a  particular  event  can  be  associated  with  varying  degrees  of  membership  in 
more  than  one  class  (the  distribution  of  membership  is  fuzzy,  rather  than  binary).  Underlying 
this  association  is  a  complete  algebra  (such  as  Boolean  Algebra)  for  deriving  logical  conclusions 
concerning  the  impact  of  a  particular  set  of  events. 

This  approach  has  been  used  commercially  to  adjust  the  focus  in  cameras,  or  to  sense  and 
respond  to  conditions  in  household  appliances. 


3.5  Expert  Systems  (ES) 

The  term  Expert  System  refers  to  a  generic  class  of  paradigms  which,  in  general,  consist  of  two 
major  components;  a  knowledge  base  and  an  inference  engine.  The  knowledge  base  contains  a 
representation  of  the  rules,  heuristics  and  experience  of  appropriate  responses  in  a  particular 
domain.  The  inference  engine  is  (or  can  be)  a  complex  procedure  for  searching  through  the 
knowledge  base  for  paths  of  interconnected  rules  leading  to  a  conclusion. 

The  knowledge  base  can  be  represented  by  a  collection  of  rules  of  the  form: 

IF  (antecedent)  TFtEN  consequence(s). 

This  collection  of  rules  is  obtained  through  a  process  referred  to  as  knowledge  acquisition,  which 
traditionally  has  meant  interviews  with  human  experts  who  enunciated  their  procedures  in  the 
form  of  rules  for  dealing  with  a  particular  situation. 

The  instantiation  of  the  rules  in  the  knowledge  base  can  proceed  in  a  haphazard  way,  in  the  sense 
that  rules  are  entered  as  they  are  thought  of  by  the  expert,  and  a  particular  situation  can  be  tested 
for  completeness  and  the  rule  base  augmented  as  gaps  are  found. 

The  inferencing  engine  can  be  designed  to  accept  an  input  state  vector  and  to  search  for  a  rule 
with  an  antecedent  which  corresponds.  The  consequence  of  this  rule  yields  the  input  to  the  next 
search,  or  the  final  conclusion.  Very  complex  inferencing  procedures  have  been  created  in  an 
attempt  to  emulate  the  reasoning;  both  straight-line  and  circular  of  the  corresponding  expert.  A 
typical  expert  system  is  divided  into  four  interacting  parts.  The  first  part  is  the  knowledge-base 
containing  the  problem's  facts  or  other  static  information  and  heuristics  possibly  in  the  form  of 
rules.  The  second  part  is  the  working  memory  containing  relevant  data  for  the  current  problem 
being  solved.  The  third  part  is  the  reasoning  method  or  inference  engine  controlling  the 
organization  of  the  problem  data  and  the  route  to  be  followed  through  the  knowledge-base.  The 
fourth  part  of  an  expert  system  is  an  interface  module  which  interacts  with  the  user  or  databases. 
Inferencing  can  be  halted  to  query  the  data  source  (human  or  otherwise)  for  clarification  or 
additional  data  if  needed  to  continue  a  line  of  reasoning. 

In  general,  there  are  two  kinds  of  reasoning  methods  used  in  expert  systems.  The  first  type 
works  from  hypothesized  conclusions  towards  known  facts  and  is  called  ’backward  chaining' 
[13].  This  reasoning  is  commonly  used  in  diagnostic  situations.  The  second  type  which  starts 
from  a  known  fact(s)  to  prove  a  conclusion  is  called  'forward  chaining.’  Forward  chaining 
corresponds  to  a  search  from  cause  to  effect,  while  backward  chaining  corresponds  to  a  search 
from  effect  to  cause  (for  example,  diagnosis). 
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If  the  rules  form  a  single  level  hierarchy,  the  rule  base  corresponds  to  a  set  of  cases,  as  in  a  case- 
based  reasoning  system.  Such  a  situation  is  a  degenerate  case  and  normally  the  rules  are  linked 
in  a  tree-like  structure  which  must  be  searched  for  answers.  The  knowledge  base  can  be  very 
large,  containing  in  practice  several  thousands  of  rules,  although  such  large  collections  do  offer 
problems  in  testing  and  maintenance. 

Very  complex  inferencing  algorithms  can  be  devised.  Beginning  with  an  initial  input  state 
vector,  the  system  can  search  for  a  suitable  rule  which  corresponds  exactly  to  the  state  vector  and 
proceed  to  explore  the  consequences.  If  the  state  vector  is  incomplete,  either  at  the  start  or 
during  the  transition  of  the  rule  base,  it  can  request  more  data  or  suggest  possible  completion 
strategies  to  obtain  the  mission  information. 

An  expert  system  becomes  a  candidate  for  a  solution  under  the  following  conditions: 


i)  Non  Algorithmic  -  solutions  are  not  known  in  advance; 

ii)  Domain  is  bounded  and  describable; 

iii)  A  recognized  and  articulate  expert(s)  is  available  who  can  recall  all  relevant  rules; 
and, 


iv) 

Advantages: 

i) 

ii) 


Expert  performs  task  in  reasonable  time  length. 

Does  not  rely  on  large  data  samples;  and, 
Explanation  of  decisions  is  available. 


Disadvantages: 


i)  Need  articulate  and  available  expert; 

ii)  Typically  12-18  months  to  develop  a  useful  expert  system; 

iii)  Maintenance  of  a  large  system  can  be  difficult; 

iv)  User  must  understand  shortcomings; 

v)  Execution  time  can  be  slow; 

vi)  Domain  specific;  and, 

vii)  Not  efficient  for  pattern  matching. 
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3.6  Machine  Learning 

Machine  learning,  has  four  main  subtechnologies:  Induction  (learning  a  general  concept  from 
viewing  both  positive  and  negative  examples);  analytic  (deductive  approaches  using  past 
experiences  to  provide  examples  and  guide  the  solution  for  a  new  problem,  i.e.,  case-based, 
analogical,  explanation  based  learning);  genetic  algorithms  (produce  mutations  of  examples  by 
selecting  the  best  features  of  a  class  and  use  a  selection  process  to  achieve  objective);  and, 
connectionism  or  neural  nets.  Cased-based  reasoning  and  neural  nets  are  probably  the  most 
widely  known  approaches  for  machine  learning  and  are  discussed  below  in  more  detail. 

It  is  important  to  understand  the  difference  between  expert  systems  and  machine  learning.  For 
the  purpose  of  this  paper,  one  important  difference  is  the  incorporation  of  new  knowledge.  In  a 
rule-based  expert  system,  the  programmer  must  analyze  all  new  knowledge  introduced  by  the 
expert  as  to  its  usefulness  and  role;  translate  it  into  a  form  acceptable  to  a  rule-based  approach; 
program  it;  and  finally,  verify  and  validate  the  new  enhanced  program.  In  a  machine  learning 
approach,  new  knowledge  is  typically  entered  as  another  set  of  examples  (knowledge).  The 
machine  analyzes  it  and  performs  program  modifications.  The  programmer  then  verifies  and 
validates  the  new  enhanced  program. 


3.6.1  Case-Based  Reasoning 

Case-based  reasoning  (CBR)  has  received  some  attention  lately  as  theoretic  concepts  are  now 
being  applied  to  real  problems  [2&3].  This  type  of  reasoning  got  its  start  with  the  theory  of 
Dynamic  Memory  Structures  [4&5]. 

Case-based  reasoning  can  be  defined  as  the  ability  to  match  the  facts  of  a  situation  against 
previous  ones  and  adapt  the  solution  to  the  new  case.  The  central  idea  behind  CBR  is  that 
reasoning  is  done  from  previous  cases  and  not  from  "first  principles,"  for  example,  conclusions 
are  drawn  from  the  available  case.  This  allows  a  problem  solving  and  reasoning  system  to  avoid 
known  errors  and  reach  a  solution  in  a  more  guided  way,  for  example,  incorporating  the  solution 
methodologies  found  successful  in  previous  cases.  The  basic  method  is  to  first  represent  cases  in 
terms  of  features  [6],  then  to  resolve  the  reasoning  approaches. 

In  a  CBR  system,  the  input  data  vector  is  most  often  a  set  of  descriptive  attributes  of  the  world. 
The  system  compares  this  vector  to  a  stored  set  of  vectors  to  determine  if  it  can  be  identified  as 
'close'  to  one  member  of  the  stored  set.  In  a  real  sense,  the  goal  is  to  determine  if  the  systems  has 
’seen'  anything  close  to  the  present  input  vector.  If  so,  then  a  known  response  can  be  executed. 
This  corresponds,  in  a  sense,  to  storing  all  the  answers;  however,  there  is  more  to  it  than  this. 

In  the  simplest  perspective,  a  CBR  system  can  be  viewed  as  a  sequence  of 

’IF  (antecedent)  THEN  (consequence)' 

statements  in  which  each  antecedent  represents  a  case  or,  it  can  be  viewed  as  the  CASE 

statement  available  in  most  programming  languages.  This  simple  construction  is  augmented  by 

an  important  extension  and  by  an  inferencing  capability  which  can  examine  the 

results  of  the  case  search  and  formulate  'intelligent'  interpretations  of  the  failure  to  find  exact 

matches. 
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A  pathological  drawback  to  such  a  system  is  the  potential  for  a  combinatorial  explosion  in  the 
number  of  combinations  of  the  set  of  input  conditions  that  have  to  be  accounted  for.  By 
introducing  a  tolerance  measure  on  the  components  of  the  input  vector,  a  range  of  input  values 
can  be  accepted  as  representing  a  case.  This  introduces  a  certain  fuzziness  into  the  acceptance 
criteria.  An  important  advantage  follows,  however,  since  depending  on  the  definition  of  the 
attribute  tolerances,  only  a  representative  set  of  state  vectors  need  be  stored.  This  representative 
set  can  be  chosen  to  reduce  the  number  of  cases  that  must  be  stored. 

CBR  systems,  can  also  have  a  reasoning  capability  beyond  simple  comparisons.  Under 
conditions  in  which  an  acceptable  match  is  not  found,  the  system  can  determine  the  closest 
matches,  and/or  determine  which  attributes  are  causing  the  mismatch.  This  information  can  be 
used  (by  another  intelligence  source;  say  an  expert  system)  to  postulate  new  cases  or  to  modify 
the  definition  of  the  attributes  to  extend  the  virtual  volume  covered  by  a  particular  case. 


CBR  systems,  therefore,  can: 

i)  Identify  classes  of  inputs  related  by  some  definition  of  a  closeness  criterion; 

ii)  Provide  reasoning  capability  about  novel  inputs;  and, 

iii)  Dynamically  define  new  cases.  New  cases  can  be  added  when  sufficient  evidence 
exists  that  might  represent  an  interesting  (i.e.,  potentially  re-occurring) 
phenomenon. 

These  systems  can  handle  qualitative  or  quantitative  data  or  mixtures  of  the  two.  Such  systems 
while  useful  in  their  own  right,  often  can  be  used  to  provide  preprocessing  of  input  data.  They 
answer  the  question  -  "Have  I  seen  this  situation  before?"  In  addition  to  a  definite  answer,  they 
can  provide  closeness  measures  which  can  be  used  to  determine  how  close  the  new  situation  is  to 
'reasonably  close'  stored  cases.  The  results  of  this  pre-processing  can  be  used  to  respond  directly 
to  the  input  excitation,  or  provide  input  to  further  processing.  The  pre-processing  will, 
hopefully,  reduce  the  search  time  of  further  processing  elements  for  an  appropriate  response. 

A  CBR  system  becomes  a  candidate  for  a  solution  to  a  problem  under  the  following  conditions: 


i)  Each  case  can  be  represented  in  terms  of  features; 

ii)  A  criterion  is  available  for  comparing  new  cases  to  existing  ones;  and, 

iii)  Rules  can  be  used  to  modify  the  attributes  of  solution. 

Advantages: 


i)  When  decision  approach  is  unknown;  and, 

ii)  Incorporates  failure  and  success  features  in  order  to  make  predictions  concerning 
viability. 


Disadvantages: 

i)  Need  a  large  and  complete  set  of  case  records;  and, 

ii)  Need  a  large  vocabulary  set. 
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3.6.2  Artificial  Neural  Networks  (ANN) 

The  study  of  ANNs  predates  all  other  forms  of  artificial  intelligence  activity.  An  artificial 
neuron  emulates  the  processing  of  the  human  neuron  by  computing  the  inner  product  of  a  set  of 
stored  weights  and  an  input  vector.  The  inner  product  is  usually  subjected  to  a  non-linear 
smoothing  function  (e.g.,  a  sigmoid).  Arrays  of  these  neurons  are  assembled  to  relate  an  input 
vector  to  a  set  of  outputs.  The  substantial  difference  between  an  ANN  and  other  forms  of 
intelligent  systems  are  routines  that  have  been  derived  that  permit  the  weights  of  each  neuron  in 
an  array  to  be  'learned'  by  repetitive  presentation  of  the  whole  set  of  inputs.  Weights  are  adjusted 
incrementally  based  on  the  error  generated  by  the  network  and  the  predetermined  correct 
response.  While  the  theoretical  basis  for  the  training  routines  is  well  developed,  in  practice  some 
uncertainty  exists  in  achieving  convergence  due  to  the  very  complex  topology  of  the  error 
surface. 

A  vast  array  of  neural  topologies  now  exist,  and  research  on  these  and  real-time  responsive 
arrays  is  currently  in  progress.  Classical  ANNs  fall  roughly  into  three  classes:  those  that  map  an 
input  vector  onto  an  appropriate  response  vector;  those  that  minimize  an  energy  function;  those 
that  can  learn  in  real-time  (dynamically).  Real  time  networks  (called  ART  Nets)  are  able  to 
dynamically  learn  new  patterns  based  on  the  repetition  in  an  input  sequence,  and  forget  patterns 
which  no  longer  occur  with  a  sufficient  repetition  rate. 

The  major  strength  of  ANNs  is  the  ability  to  learn  the  mapping  between  the  input  vectors  and  the 
desired  output  vectors.  This  ability  to  learn  means  that  the  relationship  need  not  be  expressible 
by  any  algorithm  or  set  of  rules.  The  draw-back  is  that  examples  of  the  input  vectors  (and  the 
correct  response)  must  be  plentiful  to  permit  convergence  of  the  training  procedure.  In  this 
sense,  an  ANN  emulates  the  more  intuitive  aspect  of  human  intelligence,  in  that  conditioned  or 
learned  responses  can  be  built  into  the  neural  network  by  repetitious  training  (much  as  with  a 
human)  without  explanations. 

With  the  exception  of  the  ART  networks,  neural  network  responses  are  fixed  at  the  conclusion  of 
training.  New  inputs  can  only  be  accommodated  by  starting  the  training  process  from  the 
beginning. 

In  some  situations,  the  output  vector  of  a  neural  network  can  be  considered  as  defining  the  case 
represented  by  the  input  vector.  Identification  of  a  case  drives  a  response  reaction.  If  a  clear 
decision  is  not  possible,  the  output  can  often  be  interpreted  as  giving  a  closeness  measure  to  the 
existing  cases.  This  is  similar  to  a  fuzzy  membership  function  and  this  information  can  be 
passed  on  to  another  paradigm  (such  as  an  ES)  for  interpretation. 

An  ANN  can  be  considered  as  a  potential  solution  for  achieving  machine  intelligence  under  the 
following  conditions: 

i)  Input  vectors  can  be  related  to  a  known  output  (e.g.,  pattern  matches);  and, 

ii)  A  large  numerical  data  set  is  available  representing  input  conditions  and  the 
appropriate  classification.  This  is  often  referred  to  as  the  training  set. 

Advantages: 


i) 

ii) 


Designated  Input-output  patterns  can  be  learned;  and. 

Multiple  patterns  matches  can  arbitrarily  partition  a  complex  space. 
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Disadvantages: 

i)  Cannot  provide  rationale; 

ii)  Data  sets  must  be  representative;  and, 

iii)  Development  time  is  difficult  to  predict. 


3.7  An  Overview  -  ANN.  CBR,  ES 

A  CBR  system  and  an  ES  have  many  similarities.  They  are  distinguished  from  a  ANN  by  one 
major  feature:  An  ANN  is  a  closed  system,  in  that  the  internal  processing  is  largely 
indecipherable,  while  the  inferencing  process  of  a  CBR  or  an  ES  can  be  supplemented  by 
external  interactions  with  the  environment.  The  similarities  on  the  other  hand  are  striking.  All 
three  attempt  to  interpret  an  input  state  vector  and  derive  some  conclusion  or  action  plan.  Thus 
functionally  they  are  the  same.  The  differences  lie  in  how  the  internal  processing  is  originally 
derived  and  in  the  mechanisms  for  arriving  at  a  conclusion. 

A  CBR  could  be  thought  of  as  a  large  CASE  statement  on  the  input  vector.  If  a  case  is  found, 
then  an  action  results.  The  immediate  extension  to  this  common  programming  idea  is  that  the 
components  of  the  state  vector  can  carry  a  tolerance,  so  that  items  in  the  state  vector  which  are 
close  (in  some  sense)  to  the  stored  case,  can  be  accepted  as  equivalent  to  the  case.  In  addition,  as 
mentioned  previously,  if  no  stored  case  is  found,  the  system  can  list  the  closest  cases  and  or 
query  the  data  source  for  variations  of  the  data,  if  appropriate. 

In  addition,  a  CBR  can  dynamically  add  new  cases  according  to  some  criteria;  a  feature  which  is 
not  possible  with  an  ES  without  human  intervention.  The  basic  weakness  of  a  CBR  system  is  the 
single  level  of  the  hierarchy  connecting  input  with  output.  This  clearly  limits  the  reasoning 
ability  of  a  case  system.  A  CBR  system  serves  a  very  useful  function  as  a  pre-processor  which 
quickly  addresses  the  question  -  "Have  I  seen  this  situation  before?  and  what  did  I  do  about  it?" 

An  ES  possesses  the  ability  to  execute  complicated  hierarchical  strings  of  reasoning  linking 
cause  and  effect  (or  vice  versa).  In  addition,  the  inference  engine  can  create  a  dialogue  with  the 
data  source,  during  the  reasoning  process  to  request  more  or  variations  of  the  data  input  which 
may  support  a  particular  path  of  thought.  The  weakness  of  such  a  system  is  that  the  rule  base 
represents  the  world  view  of  the  system  and,  therefore,  must  be  complete  and  reflect  all  possible 
outcomes  of  results  of  the  state  vector,  which  suggests  that  the  outcome  must  have  been  foreseen 
and  the  appropriate  rules  embedded  in  the  system. 


3.8  Vision  Systems 

Vision  systems  are  often  considered  as  a  separate  activity  in  the  pursuit  of  artificial  intelligence. 
Robotics  vision  systems  generally  includes  any  sensor  information  designed  to  develop  and 
maintain  a  (geometric)  knowledge  base  for  path  planning,  collision  avoidance  and  task  planning. 
Sensor  information  can  vary,  depending  on  the  task  and  the  environment,  from  a  dense  pixel 
representation  of  the  environment  obtained  by  standard  television  or  infrared,  to  radar,  laser, 
and/or  sonar  range  information,  or  Doppler  scanners. 

The  task  is  to  interpret  the  available  information  and  respond  with  a  guidance  plan  which 
accounts  for  the  perceived  terrain  and  the  original  goal  of  the  mission. 
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One  approach  to  obtaining  a  view  of  the  world  is  mult  ire  solution  image  processing  as  proposed 
by  JPL  [7].  This  system  illustrates  a  solution  to  minimizing  the  processing. 

The  system  begins  with  an  image  of  512  x  512  pixels,  then  successively  selecting  every  other 
pixel  to  create  subsampled  images  at  256  x  256,  etc.,  down  to  16  x  16  pixels.  The  goal  is  to 
process  images  at  the  lowest  possible  resolution  for  the  given  task.  The  results  of  this  lower 
level  processing  is  used  to  guide  a  "window  of  attention"  to  those  parts  of  the  higher  resolution 
images  that  need  to  be  analyzed  in  detail.  Special  purpose  processors  could  be  assigned  to  each 
window  to  achieve  real-time  response. 

The  software  tools  developed  at  JPL  include  those: 

i)  For  manipulating  image  pyramids; 

ii)  For  spawning  windows-of-attention  based  on  a  variety  of  cues; 

iii)  For  creating  new  pre-attentive  cues  for  spawning  windows-of-attention; 

iv)  For  analyzing  the  windows  for  motions,  texture,  and  other  properties; 

v)  Deciding  when  and  how  to  move  the  window-of-attention  to  a  lower  (or  higher) 
array;  and, 

vi)  For  matching  window  with  other  templates. 

The  details  of  the  software  paradigms  are  not  given  in  the  paper,  however,  for  our  purposes  it  is 
clear  that  several  things  have  been  accomplished: 

i)  A  sensor  system  has  been  adopted  to  produce  a  pixel  stream  representing  the 
world; 

ii)  The  pixel  stream  has  been  processed  (algorithmically)  to  create  a  pyramid  of 
images  of  lowering  resolution;  and, 

iii)  A  host  of  software  routines  have  been  set  up  to  derive  appropriate  information 
from  scanning  the  images  and  moving  across  the  images  as  seen  to  be  necessary. 

The  JPL  system  was  designed  to  interact  with  an  operator  (telerobotics)  for  guidance  in  feature 
and  object  tracking,  as  well  as,  for  scene  segmentation  and  object  modeling.  It  seems  a  small 
step  to  create  surrogates  of  the  operator's  experience  and  knowledge  and  create  a  fully 
autonomous  system. 
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4.0  HYBRID  SYSTEMS 
4.1  Introduction 

A  hybrid  artificial  intelligence  architecture  can  be  constructed  by  combining  one  or  more 
artificial  intelligence  technologies  such  as  expert  systems  and  machine  learning.  Many  systems 
typically  require  more  than  one  AI  technology  to  successfully  represent  the  knowledge  and 
implement  the  required  reasoning  abilities. 

It  is  clear  that  the  emulation  (in  software)  of  human  intelligence  will  require  the  creation  of 
various  kinds  of  responses  to  the  environment.  Our  proposal  is  that  combinations  of  the  various 
paradigms  operating  cooperatively  in  a  hybrid  system  will  be  the  final  answer  to  achieving 
intelligence  in  machines. 

In  this  section  we  propose  a  model  of  an  intelligent  machine  with  a  partition  of  capabilities 
deemed  necessary  to  achieve  human-like  responses  to  new  situations.  The  model  accounts  for 
rational,  intuitive  and  reactive  responses. 


4.2  A  Brain  Model  of  a  Hybrid  System 

A  human  is  alleged  to  have  a  brain  with  the  rational  function  largely  performed  on  one  side  and 
the  more  intuitive  functions  on  the  other.  In  addition  we  have  many  reactionary  type  functions 
performed  by  the  spine  and  by  lower  level  reptilian  portions  of  our  total  brain.  If  this  partition  of 
the  brain  has  been  favoured  by  evolution,  there  is  perhaps  some  benefit  to  exploring  the  need  for 
such  partitions  in  creating  an  emulation  of  human  intelligence. 

In  terms  of  the  attributes  ascribed  to  the  various  paradigms  in  the  preceding,  it  is  clear  that,  in  a 
broader  sense,  the  various  AI  paradigms  can  be  grouped  into  similar  brain-like  functional 
divisions. 

Algorithms  clearly  correspond  to  the  reactive  portion  of  our  brain  functions.  CBR  and  ES 
correspond  to  our  capability  for  logical  inferencing  based  on  knowledge,  acquired  from 
experience  and/or  other  instruction,  which  may  not  always  be  intuitively  obvious.  ANNs 
correspond  to  our  intuitive  capability  for  'sensing'  that  some  conclusion  follows  from  an 
observation  that  may  not  be  derivable  as  a  logical  inference. 

It  seems  reasonable  to  suggest  that  guidance  and  control  systems  with  a  wide  range  of 
capabilities  must  emulate  these  three  capabilities  if  they  are  to  respond  to  a  variety  of  conditions 
through  out  their  task  assignment. 

If  such  is  the  case,  then  clearly  there  is  a  need  for  a  methodology  for  specifying,  designing, 
implementing,  testing  and  maintaining  such  a  system.  We  have  proposed  such  a  methodology 
[8],  however  it  is  beyond  the  scope  of  this  presentation.  It  is  worthwhile  to  support  this 
conclusion  by  examining  some  current  work  in  developing  hybrid  AI  systems. 
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5.0  HYBRID  INTELLIGENT  SYSTEMS 
5.1  Introduction 

In  this  section,  we  will  consider  the  integration  of  ES  and  ANNs  to  create  complementary  hybrid 
architectures.  These  two  paradigms  are  sufficiently  mature  to  be  serious  contenders  for  some 
decision  processes  in  a  G&C  system.  Integrating  CBR  and  ANNs  impose  the  same  set  of 
problems. 

The  direct  integration  of  neural  networks  and  expert  systems  is  not  straight-forward  for  a  variety 
of  reasons: 

i)  The  nature  of  the  data  in  a  neural  network  is  numeric,  while  in  an  expert  system  it 
is  symbolic; 

ii)  Relations  in  neural  networks  are  causal  and  represent  associations,  while  in  expert 
systems  relations  are  structured  and  allow  combinations;  and, 

iii)  The  characteristic  behavior  in  a  neural  network  is  self-organization,  while  in 
expert  systems  the  combinatorial  capability  of  structures  is  the  major  feature. 

These  distinctions  characterize  and  emphasize  the  differences  in  the  two  approaches,  however, 
they  also  provide  clues  as  to  how  they  can  be  complementary.  Integrating  the  capabilities  of  the 
two  paradigms  involve  the  preliminary  partition  of  the  original  problem,  the  sharing  of  data,  and 
the  global  control  of  interaction. 

There  are  two  basic  integration  approaches: 

i)  Autonomous  Functional  Units:  The  most  general  architecture  would  be 
composed  of  an  interconnection  of  functional  units  each  chosen  to  optimize  the 
performance  based  on  the  type  of  decision  process  to  be  executed;  and, 

ii)  Embedded  Sub-Systems:  The  simplest  architecture  from  a  control  point  of  view 
is  to  embed  one  system  within  the  other.  In  this  way  the  embedded  system  is 
called  when  needed  to  supplement  the  capabilities  of  the  other.  Typical 
structures,  for  example,  could  include  an  expert  system  which  interprets  the 
results  of  a  neural  network,  or  a  neural  network  called  to  analyze  or  preprocess  a 
data  set  for  an  expert  system. 

The  Autonomous  Functional  Units  architecture  offers  the  greatest  potential  for  implementing  a 
wide  variety  of  problems,  however,  it  also  offers  the  greatest  challenge  in  design,  implementation 
and  testing.  Such  an  organization  requires  an  explicit  or  implicit  control  mechanism  which 
directs  the  propagation  of  decisions  through  the  various  functional  units.  This  control  can  be 
established  by  a  separate  task-allocation  or  by  the  fixed  interconnection  of  the  units. 

There  are  two  important  considerations  which  distinguish  integrated  systems  from  others: 
Representation  and  Global  Inferencing. 
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5.2  The  Representation  Issue 

Three  approaches  have  been  used  for  representing  an  integrated  expert  system/neural  network: 

i)  Data  Interface:  The  easiest  is  to  represent  the  individual  functions  independently 
and  to  communicate  by  means  of  a  data  interface.  This  reduces  the  integration  of 
the  data  requirements  for  each; 

ii)  Node  Links:  This  involves  linking  a  component  of  the  expert  system  knowledge 
base  (e.g.,  a  fact,  a  frame,  or  an  object)  with  a  node  in  the  neural  network.  This 
allows  both  systems  to  access  information  simultaneously;  and, 

iii)  Connectionist:  The  expert  system  becomes  a  node  in  the  neural  network.  An 
update  mechanism  is  responsible  for  maintaining  a  consistent  knowledge 
representation,  and  for  ensuring  other  components  are  aware  of  the  change. 

Updating  mechanisms  have  two  forms: 

i)  Synchronous:  Regularly  updates,  based  on  a  clock  signal;  and, 

ii)  Asynchronous  fas-needed):  The  knowledge  representation  of  an  object  or 
component  is  updated  only  when  that  component  is  queried.  This  is  obviously 
less  resource-utilization  intensive  than  a  synchronous  update. 

Consistency  management  determines  the  current  correct  value  of  a  symbol/node.  Consistency 
management  acts  as  a  mediator  between  the  two  systems.  This  is  obviously  important  if  the 
possibility  exists  for  disagreement  at  any  point  in  the  over-all  inferencing  process.  There  are 
three  common  approaches: 

i)  Recency:  The  manager  takes  the  value  that  is  most  current,  regardless  of  which 
system  it  is  from; 

ii)  Confidence:  The  manager  compares  the  level  of  activation  on  the  neural  network 
node  to  the  level  of  certainty  generated  by  the  expert  system,  and  selects  the 
highest  value;  and, 

iii)  A-priori:  The  consistency  is  based  on  always  accepting  either  the  expert  system 
or  the  neural  network. 


5.3  The  Inferencing  Issue 

Global  inferencing  in  an  integrated  system  is  either  centrally  controlled  or  is  distributed: 

i)  Central  Control:  This  implies  a  single  module  responsible  for  guiding  the 
inferencing  process.  An  expert  system  can  be  designed  to  switch  between 
modules  depending  on  their  responses  or  other  performance  criteria,  or  a  neural 
network  can  be  used  to  focus  the  search  and  choose  the  next  module;  and, 

ii)  Distributed:  This  implies  that  control  is  based  on  some  form  of  interaction  either 
based  on  the  interconnections  or  on  some  interaction  guidelines. 
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Several  different  systems  for  the  classification  of  hybrid  architectures  of  neural  networks  and 
expert  system  have  been  proposed.  Among  them,  the  system  proposed  by  Caudill  [9]  and  the 
system  proposed  by  Medsker  &  Bailey  [10],  are  typical  and  represent  different  views  of  the 
classification  of  the  ES-NN  hybrid  architectures. 

Caudill's  system  (the  CD  system)  includes  five  classes  of  the  ES-NN  hybrid  architectures: 
formal  separation;  buried  networks;  explanation  by  confabulation;  digging  for  secrets  and  two- 
step  tango.  In  the  CD  system,  the  classification  is  primarily  concerned  with  the  class  differences 
in  the  functionalities  of  expert  systems  and  neural  networks. 

Medsker  and  Bailey's  system  (the  MB  system)  also  includes  five  classes  of  the  ES-NN  hybrid 
architectures:  stand-alone;  transformational;  loosely-coupled;  tightly-coupled;  and,  fully- 
integrated.  In  the  MB  system,  the  class  boundaries  are  not  well  defined.  In  some  classes,  the 
classification  of  the  ES-NN  hybrid  architectures  is  appropriate  based  on  the  characteristics  of 
their  software  architecture,  however,  is  inappropriate  in  their  functionality  characteristics. 

In  a  complete  classification  system,  the  functionalities  of  expert  systems  and  neural  networks  in 
the  hybrid  architectures  and  the  characteristics  of  software  architecture  should  all  be  considered. 
To  make  up  the  deficiencies  of  the  CD  system  and  the  MB  system,  we  proposed  a  new 
classification  system  of  the  ES-NN  hybrid  architectures  (the  CES  system).  The  CES  system 
includes  seven  classes:  autonomous  functional  units;  closed  embedded  sub-systems;  open 
embedded  sub-systems;  simulating  externally;  simulating  internally;  transformational  systems; 
and,  developmental  tools.  In  defining  these  classes,  we  considered  the  difference  in  the 
functionality  of  expert  systems  and  neural  networks  in  the  ES-NN  hybrid  architectures  as  well  as 
the  characteristics  of  software  architecture. 

The  seven  classes  will  be  defined  in  the  next  seven  sections.  For  each  class,  its  application, 
advantages  and  disadvantages  or  limitations  will  be  discussed. 


5.3.1  Autonomous  Functional  Units 

This  class  is  the  mainstream  in  the  study  of  the  hybrid  architectures  of  expert  systems  and  neural 
networks.  In  this  class,  each  expert  system  or  neural  network  is  an  autonomous  functional  unit. 
In  a  decision  procedure,  each  decision  node  is  implemented  by  an  expert  system  or  a  neural 
network  independently.  This  class  is  applicable  for  the  cases  where  a  large  and  complex  problem 
can  be  broken  into  small  sub-problems;  each  sub-problem  would  be  solved  by  individual  expert 
systems  or  neural  networks. 

A  classic  delivery  system  problem  provides  a  good  example  to  demonstrate  how  the  problem  can 
be  solved  using  the  autonomous  functional  units  technique.  The  problem  is  to  create  a  system 
that  can  schedule  truck  deliveries  for  customers.  This  problem  includes  two  distinct  sub¬ 
problems:  which  packages  should  go  on  which  track;  and,  what  route  each  truck  should  take  for 
the  most  efficient  delivery  path. 

The  first  sub-problem  is  primarily  concerned  with  how  to  reasonably  assign  a  package  to  a  truck, 
based  on  the  destination,  weight  and  size  of  the  package,  the  characteristics  of  other  packages, 
and  the  characteristics  of  various  trucks.  The  expert  system  is  chosen  since  the  package- 
assignment  rules  are  not  difficult  to  articulate. 
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The  second  sub-problem  is  optimization,  in  which  the  best  path  for  a  given  set  of  stops  must  be 
determined.  This  kind  of  challenge  is  not  easily  solved  by  a  set  of  rules  as  there  exists  the  vast 
number  of  possible  destination  combinations.  On  the  other  hand,  certain  neural  networks  deal 
easily  with  such  optimizations.  They  can  generally  provide  a  nearly  optimal  path  and  do  so 
quickly. 

This  example  shows  an  excellent  decomposition  of  the  overall  problem  and  an  optimal  functional 
allocation  between  an  expert  system  and  a  neural  network.  Using  each  kind  of  system  to  tackle 
the  parts  of  the  problem  to  which  it  is  most  suited,  provides  both  relatively  easy  implementation 
and  reliable  operation. 

In  implementing  the  hybrid  architectures  of  this  class,  the  data  coupling  between  an  expert 
system  and  a  neural  network  is  important,  since  it  affects  the  efficiency  and  performance  of  the 
final  system.  There  are  basically  two  coupling  approaches:  loose  coupling;  and,  tight  coupling. 


5.3. 1.1  Loose  Coupling 

The  loose  coupling  approach  is  characterized  by  a  data  sharing  mechanism  involving  external 
data  structures.  The  storage  of  intermediate  data  and  its  formatting  for  use  by  the  next  process  is 
done  in  the  data  structure. 

This  coupling  approach  offers  several  benefits  to  the  integration  of  expert  systems  and  neural 
networks: 

i)  They  are  relatively  easy  to  design  and  develop; 

ii)  They  are  amenable  to  the  use  of  commercial  development  platforms;  and, 

iii)  They  are  easy  to  maintain. 

There  are,  however,  three  major  limitations  to  this  approach: 

i)  There  is  often  a  great  deal  of  redundancy  in  the  development  process; 

ii)  The  operating  time  is  usually  longer,  because  of  the  required  interfaces  between 
the  sub-systems;  and, 

iv)  There  is.  a  high  communications  overhead. 


5.3. 1.2  Tight  Coupling 

Expert  system  and  neural  network  share  common  memory  resident  data  (instead  of  passing  data 
structures  or  using  external  data  files).  This  improves  the  interactive  capabilities  (compared  to 
the  loose  coupling  approach)  and  enhances  performance.  Tightly  coupled  systems  often  appear 
to  have  the  same  architecture  as  a  loosely  coupled  system,  however,  if  properly  designed,  they 
are  faster  because  data  is  shared  directly  (without  intermediate  external  data  structures  or 
transformations). 


The  tight  coupling  approach  offers  several  benefits  to  the  integration  of  expert  systems  and 
neural  networks: 

i)  They  have  reduced  communications  overhead,  and  improved  run-time 
performance  (compared  to  loosely  coupled  systems); 

ii)  Several  commercial  development  platforms  are  available  (e.g.,  Gensym's 
Neuronline);  and, 

iii)  They  offer  design  flexibility  and  robust  integration. 

There  are  three  principle  limitations: 

i)  The  development  and  maintenance  complexity  increases  due  to  the  internal  data 
interface; 

ii)  The  system  suffers  from  redundant  data  gathering  and  processing  due  to  the 
independence  of  the  two  systems  (similar  to  loose  coupling);  and, 

iii)  The  verification  and  validation  process  is  more  difficult  (particularly  for 
embedded  systems). 


5.3.2  Closed  Embedded  Sub-Systems 

In  this  class,  one  system  is  embedded  within  the  other  to  implement  some  of  the  functions.  Of 
the  latter,  however,  the  embedded  system  is  transparent  to  the  user.  In  a  sense,  this  class  contains 
the  embedded  sub-systems  architectures  which  are  strictly  defined,  compared  with  the  embedded 
sub-systems  architectures  described  in  later  which  are  loosely  defined. 

There  are  two  known,  typical  approaches  in  this  class:  a  neural  network  as  the  rule  firing 
mechanism  in  an  expert  system;  and,  neural  networks  as  the  knowledge  base  in  expert  systems. 


5. 3.2.1  Neural  Networks  as  the  Rule  Firing  Mechanism  in  Expert  Systems 

Consider  the  role  of  the  inference  engine  in  the  rule-based  system.  During  execution  it  must 
perform  at  least  three  tasks:  match  the  state  of  the  world  to  the  IF-clauses  of  the  production  rules 
in  the  rule  base;  select  one  or  more  of  these  rules;  and,  fire  the  rule  while  computing  the 
consequences  of  the  firing.  This  match-select-fire  operation  is  a  minimal  description  of  a 
garden-variety  expert  system.  But  the  first  step  in  the  process  is  to  match  the  state  of  the  world 
to  the  conditional  clauses  of  the  rules,  more  specifically  to  pattern-match;  and  pattern-matching 
is  exactly  what  neural  networks  do  best. 

The  basic  idea  is  to  bury  a  neural  network  inside  the  inference  engine  of  a  rule-based  system  and 
use  the  neural  network  to  perform  highly  efficient  and  intelligent  selection  of  the  next  rule  to  fire. 
The  obvious  benefit  would  be  that  the  resulting  expert  system  would  run  faster  and  select  more 
intelligently  the  correct  rule  to  fire  in  cases  of  multiple  matches. 

The  approach  would  benefit  very  large  rule  bases  in  which  sequential  rule-matching  can  take  a 
long  time,  and  in  systems  that  frequently  have  several  rules  that  match  the  current  state.  The 
rule-based  system  would  leverage-off  the  parallel  nature  of  the  neural  network  to  improve  its 
efficiency  and  performance. 
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53.2.2  Neural  Networks  as  the  Knowledge  Base  in  Expert  Systems 

One  of  strengths  of  neural  networks  is  their  associative  memory  capability,  which  can  be  utilized 
to  build  the  knowledge  base  in  an  expert  system.  The  neural  network  memorizes  the  facts  during 
its  training.  Within  the  expert  system,  the  neural  network's  input  and  output  nodes  are  used  as 
the  outlets  of  the  facts.  Each  instance  of  the  pair  of  input  and  output  patterns  provides  a  fact  to 
the  expert  system. 

The  embedded  neural  knowledge  base  efficiently  and  compactly  stores  a  large  volume  of  facts. 
However,  the  issue  of  knowledge  base  updating  in  such  an  integration  architecture  can  be 
difficult,  depending  on  what  type  of  neural  network  is  used.  If  the  incremental  retraining 
(accepting  new  facts,  with  no  need  of  retraining  all  previously  accepted  facts)  is  possible,  the 
incremental  updating  ability  of  the  neural  knowledge  base  clearly  is  an  advantage  of  the 
integration  architecture.  If  the  incremental  retraining  is  not  allowed  (e.g.,  in  the  backpropagation 
neural  net),  this  becomes  an  obvious  drawback. 

Applications  of  this  technique  include  diagnosis,  pattern  recognition  and  classification. 


5.3.3  Open  Embedded  Sub-Svstems 

In  this  class,  one  system  is  used  as  the  main  system,  the  other  is  used  as  the  secondary  system  to 
be  called  when  needed  to  improve  the  performance  of  and/or  to  supplement  the  capabilities  of 
the  main  system.  Both  the  main  system  and  the  secondary  system  are  invisible  to  the  user.  This 
class  differs  from  the  autonomous  functional  units  class  in  that  the  hybrid  architecture  of  this 
class  only  implements  one  node  in  a  decision  procedure.  It  is  considered  that  this  class  contains 
the  loosely-defined  embedded  sub-systems  architectures,  compared  with  the  strictly-defined 
embedded  sub- systems  architectures. 

Typical  hybrid  architectures  in  this  class  include  an  expert  system  used  to  augment  the 
explanation  facility  of  a  neural  net. 


5.3.4  Expert  Systems  Interprets  The  Results  of  Neural  Networks 

In  the  complete  system,  an  input  is  presented  to  the  network  and  the  network  provides  the  correct 
response  during  normal  operation.  When  the  user  asks  for  an  explanation,  the  expert  system 
considers  the  original  input  pattern  as  well  as  the  neural  network's  answer.  It  then  backward 
chains  from  the  answer  to  the  input  pattern,  constructing  a  plausible  chain  of  reasoning  to  explain 
why  this  answer  may  be  correct.  In  essence,  it  rationalizes  an  explanation  on  the  spot,  making  it 
correspond  with  the  neural  network's  response. 

This  sort  of  rationalization  response  provides  the  user  with  reassurance  that  the  neural  network  is 
operating  properly  without  relying  on  the  rule-based  system  to  generate  the  solution.  The  rule- 
based  system  does  not  have  to  be  detailed  or  complete;  it  only  has  to  be  complete  enough  to 
justify  the  answer  to  the  user.  This  sort  of  post-response  rationalization  is  probably  a  lot  closer 
to  the  way  people  solve  problems  than  we  would  like  to  admit. 


2-19 


5.3.5  Simulating  Externally 

In  this  class,  a  hybrid  architecture  appears  to  be  a  network.  Internally,  each  node  of  the  network 
is  a  symbolic  information  processing  mechanism,  which  can  be  as  simple  as  a  concept  node  in 
the  knowledge  structure  built  in  an  expert  system,  or  as  complex  as  a  complete  expert  system. 
Externally,  the  network  is  a  mimic  of  a  neural  network,  two  nodes  communicates  through  the 
propagation  of  numeric  information  along  the  interconnection  between  them.  Connectionist 
expert  system  systems  [11]  and  macro-connectionist  distributed  knowledge-based  system  [12] 
are  two  good  examples  of  such  a  hybrid  architecture. 


5.3.6  Connectionist  Expert  Systems 

Connectionist  expert  systems,  in  general,  rely  on  local  knowledge  representations,  as  opposed  to 
the  distributed  representation  of  most  neural  networks,  and  they  reason  through  spreading 
activation.  Connectionist  expert  systems  represent  relationships  between  pieces  of  knowledge 
with  weighted  links  between  symbolic  nodes.  Such  systems  have  been  used  in  medical 
diagnosis,  information  retrieval  and  analysis,  and  pattern  classification. 

The  potential  benefits  of  such  a  system  include: 

i)  Robustness,  improved  performance,  and  increased  problem  solving  capabilities. 
The  robustness  and  performance  improvements  derive  from  the  dual  nature  of  the 
knowledge  representations  and  data  structures;  and, 

ii)  A  properly  designed  integrated  architecture  can  yield  a  broad  range  of 
functionality  -  such  as  adaptation,  generalization,  noise  tolerance,  justification, 
and  logical  deduction. 

There  are  several  limitations: 

i)  There  is  some  complexity  involved  in  specifying,  designing  and  building  fully- 
integrated  models; 

ii)  There  are  no  development  tools;  and, 

iii)  Verification,  validation  and  maintaining  such  systems  is  difficult. 


5.3.7  Simulating  Internally 

In  contrast  with  the  simulating  externally  class  in  which  the  neural  network's  external  attributes, 
the  representation  and  global  inferencing  mechanisms  are  simulated,  the  simulating  internally 
class  is  concerned  with  the  neural  network's  internal  representation. 

Basically,  the  way  to  find  out  the  internal  representation  of  a  neural  net  is  to  first  train  the  neural 
network  to  solve  the  problem  and  then  reverse-engineer  it  to  determine  the  features  the  neural 
network  is  using  to  make  decisions.  For  example,  in  the  training  of  a  neural  network  for  an 
alphabet-recognition  task,  we  could  force  some  of  the  neurons  in  the  middle  layer  to  look  for 
obvious  features  like  vertical  straight  lines  or  curves  of  various  sizes  and  locations.  These 
neurons  then  become  the  designated  feature  detectors.  By  analyzing  the  behavior  of  these 
feature  detectors,  it  may  be  possible  to  determine  how  the  neural  network  makes  decisions. 
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There  are  basically  two  benefits  from  doing  so: 

i)  Access  to  more  detailed  information  on  how  the  neural  network  makes  decisions, 
which  is  obviously  useful  when  the  neural  network  must  be  able  to  explain  what  it 
does;  and, 

ii)  Study  the  nature  of  distributed  representations  and  how  the  brain  might  process 
and  store  information. 

There  are  several  limitations: 

i)  There  is  no  guidelines  on  how  to  train  a  neural  network  to  set  up  feature  detectors 
in  the  middle  layer.  Even  if  these  guidelines  existed,  the  control  of  this  type  of 
training  may  be  tedious;  and, 

ii)  The  post-analysis  of  the  behavior  of  the  feature  detectors  may  be  difficult. 

The  applications  of  this  hybrid  architecture  class  include  image  processing,  feature  extraction 
and  decision  making. 


5.3.8  Transformational  Systems 

The  expert  system  is  transformed  into  a  neural  network  or  vice  versa.  A  typical  situation  might 
be  that  the  development  system  was  chosen  as  a  neural  network  because  the  application  was 
data-intensive  and  there  was  a  need  to  generalize  and  filter  errors  in  the  data.  The  delivery  of  an 
expert  system  was  chosen  because  of  the  need  to  document  and  verify  the  knowledge  used  to 
make  the  decisions,  and  justification  of  the  decision  was  needed.  On  the  other  hand,  an  expert 
system  could  be  converted  to  a  neural  network  because  it  was  incapable  of  solving  the  whole 
problem,  or  the  speed,  adaptability,  and  robustness  of  the  neural  network  is  required.  Knowledge 
from  the  expert  system  is  used  to  set  the  initial  conditions  and  to  derive  the  training  set  for  the 
neural  network. 

Transforming  one  system  into  another  can  offer  several  benefits: 

i)  They  are  often  quick  to  develop  and  require  maintenance  of  only  one  system; 

ii)  Development  occurs  in  the  most  appropriate  environment;  and, 

iii)  The  delivery  system  is  the  most  appropriate  to  the  operational  needs. 

Limitations  of  this  approach  are  two-fold: 

i)  There  is  no  automated  means  of  effecting  the  transformation;  and, 

ii)  There  is  no  known  method  for  accurately  and  completely  performing  the 
transformation. 

Despite  these  limitations,  this  approach  has  been  used  many  times,  which  suggests  that  with 
reasonable  systems  such  transformations  are  possible.  The  question  becomes  "Which  system 
should  be  used  for  development  and  which  for  operation?" 
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5.3.9  Developmental  Tools 

In  this  class,  one  system  is  used  to  assist  the  development  of  the  other.  There  are  basically  two 
approaches:  neural  networks  extract  expert  system  rules;  and,  expert  systems  control  neural 
network  training. 


5.3.10  Neural  Networks  to  Extract  Expert  System  Rules 

First,  a  neural  network  is  trained  to  solve  the  problem  at  hand.  Then,  the  trained  neural  network 
is  used  to  generate  a  set  of  rules  based  on  the  given  input  data.  Finally,  the  generated  rule-set  is 
used  to  construct  the  rule-base  of  an  expert  system. 

In  effect,  the  neural  network  is  trained  to  be  the  expert  in  the  expert  systems  development.  The 
obvious  use  of  this  procedure  is  for  problems  where  no  expert  exists  to  construct  a  rule-based 
system,  but  where  the  properties  of  a  rule-based  solution  are  desired.  Alternatively,  this 
approach  can  be  used  to  supplement  a  human  expert  who  is  having  trouble  articulating  the 
domain  rules. 

This  process  has  the  obvious  drawback  of  requiring  the  developers  to  go  through  the 
development  procedure  for  both  the  neural  network  and  expert  system.  Nevertheless,  in  some 
cases,  this  technique  can  be  very  useful  for  helping  an  expert  articulating  rules,  providing  an 
expert  with  a  set  of  sample  rules  that  can  be  edited  as  appropriate  and  substituting  for  an 
unavailable  expert. 


6.0  SUMMARY 

It  has  been  proposed  that  fully  autonomous  robotics  systems  possessing  intelligent  guidance  and 
control  capabilities  will  necessarily  utilize  'intelligent'  software  paradigms.  Further,  it  is 
suggested  that  these  paradigms  will  be  used  in  hybrid  combinations  to  simulate  the  various 
capabilities  of  the  human  brain. 

A  hybrid  brain  model  was  proposed  with  several  of  the  well  known  paradigms  allocated  to  one  of 
three  partitions.  This  or  some  model  like  it  must  eventually  provide  the  intelligence  capabilities 
of  an  intelligent  robot.  What  is  needed  is  a  complete  methodology  for  deriving  the  appropriate 
allocation  of  functions  to  the  available  paradigms,  and  a  means  of  communication  between  each. 
Evidence  is  already  appearing  in  the  literature  of  the  use  of  such  systems  and  it  is  anticipated  that 
they  will  become  increasingly  common  over  the  next  few  years.  What  has  failed  to  appear  yet  is 
a  suitable  computer-aided  system  engineering  (CASE)  platform  for  developing  each  paradigm 
and  for  linking  and  testing  the  result. 

Let  us  conclude  this  portion  of  the  presentation  with  some  philosophical  comments  on  machine 
intelligence  derived  from  'intelligent'  software.  An  intelligent  machine  is  capable  of  more  than 
an  appropriate  response  to  a  predefined  situation;  intelligence  in  machines  (and  humans)  is 
usually  thought  of  as  the  ability  deal  with  variations  on  a  set  of  situations,  and  the  ability  to 
reason  about  the  most  appropriate  response  to  new  situations.  In  the  limit,  perhaps,  intelligence 
is  the  ability  to  carry  out  a  mission  given  unforeseen  circumstance,  and  to  survive  under  these 
circumstances,  even  if  the  mission  is  aborted.  The  human  race  may  already  be  undergoing  such  a 
test  of  intelligence. 


3-1 


MACHINE  INTELLIGENCE- 

THE  KEY  TO  GUIDANCE  AND  CONTROL  IN  ROBOTICS 

by 

B.  Archie  Bowen  and  David  G.  Bowen 
CompEngServ  Ltd. 

Ottawa,  Ontario,  Canada 
K1Y  1X4 


PART  B  -  EXPECTATIONS  AND  PROJECTIONS 


1.0  INTRODUCTION 

In  this  part  we  wish  to  accomplish  four  goals:  to  review  some  examples  of  hybrid  systems;  to 
illustrate  a  generic  G&C  architectural  model;  to  point  out  how  current  and  future  research  will 
impact  the  capabilities  of  G&C  software  and  finally  to  suggest  an  important  distinction  between 
military  and  civilian  requirements  in  this  area.  The  later  item  will  be  of  particular  importance  to 
those  formulating  research  projects  and  to  those  in  a  position  to  fund  them. 


2.0  PROGRESS  IN  HYBRID  SYSTEMS 

The  literature  contains  many  examples  of  hybrid  distributed  architectures  which  fall  roughly  in 
the  proposed  classes.  In  the  following  sections  several  will  be  chosen  to  illustrate  some  of  the 
categories. 


2.1  A  Resource  Projection  System 

Hanson  and  Brekke  [14]  developed  a  system  for  projecting  manpower  and  resource  requirements 
for  maintaining  networks  of  workstations  at  NASA.  The  overall  function  of  the  system  was  to 
estimate  the  resource  requirements  and  the  completion  time  for  the  services  requested  by 
customers.  A  service  request  input  is  presented  as  a  list  of  activities  required.  The  output  of  the 
system  is  in  the  form  of  spreadsheet  files,  in  which  for  each  activity  required,  the  manpower  and 
resource  allocated  and  the  date  the  jobs  to  be  completed  are  presented. 

The  major  part  of  the  required  output  is  the  allocation  of  resources,  which  can  be  obtained  using 
an  expert  system.  An  expert  system  can  properly  allocate  the  manpower  and  resource  for  the 
service  requested,  based  on  the  knowledge  (rules)  acquired  from  managers  and  technicians. 
However,  another  part  of  the  output  is  an  estimation  of  the  completion  time  for  the  activities 
required,  which  will  be  more  easily  obtained  by  using  a  neural  network.  A  neural  network  can 
be  trained  using  historical  data,  in  which  the  time  needed  for  different  types  of  activities  in  the 
past  services  are  present.  Thus,  the  complete  system  output  can  be  obtained  using  an  expert 
system  plus  a  called-up  neural  network. 

In  the  operation  of  the  system,  a  service  request,  in  the  form  of  a  list  of  activities  from  a  Service 
Request  Data  Base  (SRDB),  is  first  preprocessed  to  produce  data  files  suitable  for  the  neural 
network  and  the  expert  system.  The  expert  system  allocates  the  resources  according  to  the 
service  requested  and,  in  the  meantime,  it  also  calls  up  the  neural  network  to  obtain  the  estimate 
of  the  completion  time  for  the  given  activity  list.  Finally,  the  expert  system  produces  the  output 
in  which  the  resource  allocation  results  and  the  estimates  of  job  completion  time  are  combined. 

This  system  belongs  to  the  closed  embedded  subsystems  class  in  the  CES  classification  system 
for  ES-NN  hybrid  architecture.  The  use  of  the  neural  network  in  this  system  is  to  provide  the 
expert  system  with  a  knowledge  base  which  stores  the  facts  "(activity,  completion  time)." 


Presented  at  an  AGARD  Lecture  Series  on  ‘Advanced  Guidance  and  Control  Aspects  in  Robotics’  .June  1994. 
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2.2  A  Process  Control  System 

A  process  control  system  called  "SCRuFFy"  [14]  was  designed  for  the  control  of  mechanical 
systems  in  which  sensor  feedback  is  needed  to  monitor  the  status  of  ongoing  processes.  The 
overall  function  of  the  system  was  to  track  changes  in  the  sensor  signal  to  detect  error  conditions 
in  system’s  operation  so  corrective  actions  can  be  taken  before  serious  errors  are  committed. 

In  general,  a  procedure  of  process  control,  based  on  the  sensory  feedback,  includes  these  tasks: 

i)  Converting  the  sensor  signals  from  analog  to  digital; 

ii)  Identifying  and  classifying  the  error  conditions  contained  in  the  received  sensor 
signals; 

iii)  Tracking  the  error  condition  changes,  which  is  needed  since  an  action  to  be  taken 
against  an  error  condition  can  be  determined  only  after  the  continuous  change  of 
such  error  condition  has  been  observed  over  time;  a  drastic  correction  action 
based  on  a  single  input  of  error  condition  change  is  not  expected,  as  it  would 
probably  damage  the  machinery;  and, 

iv)  Making  a  decision  to  determine  which  action  should  be  taken  to  correct  the  error 
condition. 

The  SCRuFFy  system  uses  a  neural  net  to  accomplish  the  error  condition  classification  task.  In 
general,  to  determine  the  situations  in  which  a  statistically-based  classification  method  (such  as 
the  backpropagation  learning)  can  be  used  the  criteria  are:  the  cost  of  observation  should  be  low; 
the  features  of  the  class  may  or  may  not  all  be  presented;  the  set  of  classes  is  known  prior  to 
classification  time;  the  classifications  are  based  on  statistical  criteria;  and,  the  hierarchical 
structure  of  the  classes  is  either  not  necessary  or  used  in  only  a  limited  manner.  The 
classification  of  the  system's  operation  (error  or  normal)  conditions  based  on  the  sensor  signals  is 
a  case  fitting  these  criteria:  sensor  data  is  obtained  cheaply  at  a  high  rate  of  speed;  features  of 
the  class  may  be  undetermined  (necessitating  a  learning  algorithm);  classification  of  the  signals 
is  desired  to  match  known  categories;  classification  is  based  on  high-order  statistical  correlations 
between  features  in  the  signal;  and,  no  hierarchical  categorization  of  the  signals  is  typically 
performed.  Thus,  the  neural  net  is  chosen  here  to  accomplish  the  task. 

The  SCRuFFy  system  uses  an  expert  system  to  accomplish  the  decision  making  task.  Making 
intelligent  decisions  based  on  the  received  sensor  signals  is  very  different  from  classifying  these 
signals.  Such  decisions  may  require  merging  a  long  time  history  of  sensor  results;  all  features  of 
a  situation  may  need  to  be  present  before  expensive  corrective  operations  would  be  authorized; 
the  set  of  all  possible  problems  may  not  be  known,  the  recognition  of  a  problem  is  often  context 
dependent;  and,  hierarchical  structure  is  often  used  in  the  decision  making  processes.  Thus,  the 
neural  networks  approach  seems  unsuited,  and  such  a  task  remains  the  domain  of  traditional  AI, 
such  as  the  expert  systems  approach. 

The  SCRuFFy  system  uses  a  symbolic  analyzer  to  accomplish  the  error  condition  change 
tracking  task.  This  symbolic  analyzer  also  is  used  to  transform  the  numerical  output  of  the 
neural  network  to  the  symbolic  data  required  by  the  expert  system,  which  is  probably  more 
important  from  an  ES-NN  integration  point  of  view.  The  symbolic  analyzer  produces  the 
symbolic  information  describing  the  error  condition  changes  over  time  and  puts  it  into  a 
blackboard  (or  working  memory).  Thus,  a  blackboard-based  expert  system  can  make  control 
decisions  based  on  this  infomiation. 
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During  operation,  the  signal  processor  converts  the  form  of  sensor  signals  from  analog  to  digital 
(numerical)  for  use  by  the  neural  network.  The  neural  network,  which  was  trained  to  identify 
error  conditions  in  sensor  signals,  indicates  whether  the  operation  mode  of  the  system  is  normal 
or  erroneous  based  on  the  output  from  the  signal  processor.  The  output  from  the  neural  network 
is  fed  into  the  symbolic  analyzer.  The  symbolic  analyzer  tracks  the  changes  in  the  error 
condition  and  converts  this  information  into  symbolic  form  for  use  by  the  expert  system  which 
prescribes  corrective  actions  against  the  error  conditions.  The  symbolic  error  condition  change 
information  is  stored  in  the  blackboard,  which  the  expert  system  uses  to  make  the  decisions  on 
corrective  actions. 

The  SCRuFFy  system  belongs  to  the  autonomous  functional  units  class  in  the  CES  classification 
system  for  ES-NN  hybrid  architecture.  The  expert  system  and  the  neural  network  in  the  system 
are  loosely  coupled,  as  they  do  not  share  common  memory  resident  data  and  the  symbolic 
analyzer  does  a  data  transformation  between  them. 


2.3  A  Hypothesis  Testing  System 

A  general  problem  solving  system  was  developed  by  Gutknecht,  Pfeifer,  and  Stolze  [15],  which 
combines  the  three  paradigms  to  reason  about  and  solve  problems,  including  neural  networks, 
expert  systems  and  case-based  reasoning.  The  overall  function  of  the  system  was  to  learn  how  to 
focus  on  a  problem  and  narrow  down  to  likely  hypotheses  and  questions  to  find  the  solution  of 
the  problem. 

This  system  was  expected  to  solve  a  problem  in  a  way  that  is  similar  to  the  way  human  experts 
operate.  A  human  expert  first  has  a  "case  base"  in  his  mind,  which  contains  his  knowledge  of 
what  hypotheses  should  be  raised  and  what  tests  should  be  done  for  evaluating  the  hypotheses. 
When  a  problem  occurs,  the  human  expert,  based  on  knowledge  stored  in  the  "case  base,"  finds 
and  rates  different  hypotheses  and  possible  tests  according  to  the  given  conditions.  Through  a 
certain  number  of  hypothesis-and-test  iterations,  the  human  expert  then  focuses  on  the  best 
course  of  action.  After  the  problem  is  solved,  his  "case  base"  is  updated  with  his  performance 
and  experience  gained  in  solving  this  new  case. 

In  the  similar  way  as  a  human  expert  solves  problems  as  described  above,  the  system  includes  a 
case-base  to  store  information  from  observations  of  human  experts'  choices  of  hypotheses  and 
tests.  This  case-base  is  updated  when  the  system  is  used  and  new  instances  of  expert 
performance  are  observed.  The  system  includes  a  neural  network  trained  to  recognize  possible 
hypotheses  and  tests  according  to  certain  conditions  of  a  given  problem.  This  neural  net  accesses 
the  case-base  during  initial  training  and  for  retraining  as  needed.  The  system  also  includes  an 
expert  system  which  judges  the  hypothesis  test  results  and  determines  the  course  of  action 
required  for  solving  the  problem.  The  expert  system  accesses  the  case-base  for  learning  new 
rules  and/or  modifying  old  ones. 

The  system  belongs  to  the  autonomous  functional  units  class  in  the  CES  classification  system  for 
ES-NN  hybrid  architecture.  The  system  provides  the  architecture  such  that  the  user  calls  to  the 
neural  network  and  the  expert  system  as  needed.  The  neural  network  and  the  expert  system  are 
tightly  coupled  in  the  system,  as  they  share  the  case  base. 


3-4 


3.0  A  GENERIC  G&C  ARCHITECTURE 

Any  system  of  guidance  and  control  is  finally  limited  by  the  information  supplied  by  the  sensors. 
It  is  always  an  assumption  that  the  more  complete  the  information  about  the  state  of  the  world 
the  more  intelligent  can  be  the  responses.  Of  course  the  effectiveness  of  the  responses  is  limited 
by  the  capabilities  of  the  actuators.  It  is  reasonable  to  conclude  that  a  complete  system  is  first 
decomposable  into  sensors,  actuators  and  an  interpretation  and  response  component. 

For  vehicles,  at  least,  the  available  control  mechanisms  can  be  as  extensive  as  those  supplied  to  a 
human  operator  with  modifications  to  accommodate  the  drive  mechanisms.  It  is  therefore  the 
sensors  and  subsequent  processing  that  usually  receives  the  most  attention. 

Determining  the  state  of  the  world  is  usually  done  by  an  array  of  sensors  measuring  different 
features,  which  are  then  combined  to  produce  a  description  of  the  world.  The  question  of  sensor 
fusion  and/or  integration  has  been  a  challenge  for  many  years.  The  major  challenge  being  to 
structure  the  data  available  from  the  sensors  to  a  data  stream  suitable  for  interpretation  and 
control  decisions. 

Data  fusion  and  integration  should  be  designed  to  present  the  appropriate  data  fields  to  the 
interpretation  and  planning  functions  of  the  robot. 

The  sensor  readings  themselves  may  be  subject  to  error  and  uncertainties  caused  by  their  intrinsic 
ability  to  represent  the  world,  and  by  inherent  uncertainties  due  to  noise.  Most  importantly,  the 
significance  of  the  readings  may  be  unknowingly  jeopardized  by  the  active  influences  of  an 
enemies  attempts  to  alter  the  appearance  of  the  world.  Such  attempts  have  a  long  history  dating 
back  to  radar  and  radio  jamming  in  World  War  2.  It  is  still  possible  to  jam  a  sensor  array  and 
nullify  its  utility.  While  effective,  such  techniques  are  primitive  compared  to  techniques 
employing  deception.  Deception  is  an  attempt  to  create  a  reasonable  but  incorrect  version  of  the 
world  state  by  influencing  the  sensors.  Many  technological  advances  are  making  this  both  more 
likely  and  more  difficult  to  succeed.  The  impact  however  motivates  the  requirement  for 
multisensor  arrays  and  very  intelligent  processing. 

There  are  four  aspects  to  data  fusion  and  integration: 

i)  Redundancy; 

ii)  Complementarity; 

iii)  Timeliness;  and, 

iv)  Cost. 

Fusion  of  data  refers  to  a  combination  of  different  sensor  information  into  one  representational 
format.  Integration  refers  to  the  (synergistic)  use  of  information  from  different  sensors  to 
accomplish  a  task  of  the  system.  Fusion  is  generally  an  algorithmic  task  while  integration 
implies  a  certain  amount  of  intelligence  in  the  procession  modules. 

Waltz  and  Buede  [16]  have  proposed  a  generic  command  and  control  architecture  that  includes 
multisensor  integration  and  fusion  functions  and  a  distributed  intelligent  architecture.  We  will 
look  at  the  intelligence  aspects  of  this  early  proposal  and  suggest  what  additional  features  could 
be  added  to  create  an  intelligent  completely  autonomous  system. 
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The  proposed  functions  of  the  four  steps  shown  in  Figure  6  are: 

i)  The  sensor  bank  collects  data  from  the  environment  and  transmits  the  data  to  the 
multi-sensor  integration  and  fusion  subsystem; 

ii)  The  multi-sensor  integration  and  fusion  subsection  fuses  and  integrates  the  data  to 
find  any  events  or  targets  of  interest.  The  fused  information  representing  the 
possible  targets  or  events  which  influence  the  current  situation  are  then  sent  to  the 
decision  support  subsection; 

iii)  The  decision  support  subsection  creates,  analyses  and  ranks  alternative  courses  of 
action;  and, 

iv)  A  human  selects  a  course  of  action  (which  could  influence  the  current  situation). 

The  system  contains  multiple  feed  back  loops  for  distributed  control.  Operation  is  initiated  with 
a  query  from  the  human  for  recommended  courses  of  action.  Based  on  some  parameters  supplied 
by  the  human  and  the  assessment  of  the  current  situation  the  system  recommends  a  course  of 
action.  It  can  query  the  multi-sensor  fusion  and  integration  unit  if  necessary. 

In  order  to  suggest  the  requirement  for  intelligent  software  it  is  worth  briefly  looking  at  the 
functions  performed  by  the  major  modules.  The  process  of  data  fusion  starts  with  each  sensor 
sending  detection  report  or  tracks  to  the  queue  from  which  they  are  transformed  to  a  common 
space-time  coordinate  system  (this  is  necessary  because  of  different  data  rates  and  time 
responses). 

The  process  of  combining  the  collected  data  to  enable  the  attributes  of  a  target  (e.g.,  its  identity, 
intent,  future  behaviour,  etc.)  to  be  inferred  is  the  core  function  of  the  multi-sensor  integration 
and  fusion  subsystem.  In  the  decision  support  system,  the  first  module  is  an  expert  system.  The 
arrival  of  new  sensor  data  causes  a  forward-chaining  process  to  search  for  the  rule  with  the 
appropriate  antecedent,  and  the  situation  data  base  being  updated  with  the  consequence. 

By  backward  chaining,  a  hypothesis  can  be  searched  for  matching  antecedents,  and  the  sensor 
management  function  can  be  directed  to  search  for  data  which  might  support  the  required 
antecedent. 

There  are  of  course  many  other  approaches  to  processing  sensor  data;  they  share  the  common 
objective  of  providing  a  reliable  data  reactor  for  interpretation  and  decision  making. 


4.0  G&C  SYSTEMS  -  STATUS  AND  REQUIREMENTS 
4.1  Introduction 

There  is  a  great  variety  of  research  projects  which  can  contribute  to  the  advancement  of  the  use 
of  intelligent  systems  in  G&C  (and  many  other)  areas  [17].  These  cover  a  broad  range  of 
activities  from  developments  in  AI  to  architectural  and  sensor  improvements.  A  brief  summary 
of  some  of  these  may  prove  useful  in  focusing  the  previous  presentation. 


3-6 


4.2  Basic  AT  Research 

Al-based  research  may  prove  especially  useful  in  areas  such  as: 

i)  Sensor  selection; 

ii)  Automatic  task  error  detection  and  recovery;  and, 

iii)  The  development  of  high-level  representations. 

Neural  net  research  will  impact  areas  such  as: 

i)  Object  recognition  through  the  development  of  distributed  representations  suitable 
for  the  associative  recall  of  multisensor  information;  and, 

ii)  The  development  of  robust  multisensor  systems  that  are  able  to  self-organize  and 
adapt  to  changing  conditions  (e.g.,  sensor  failure). 


4.3  At  the  Architectural  Level 

The  whole  guidance  and  control  problem  should  be  reduced  to  a  common  architectural  function 
with  the  algorithmic,  and  intelligent  software  portions  optimally  allocated.  The  design  of  the 
functional  requirements  in  such  a  format  that  allocations  to  the  appropriate  paradigms  become 
obvious  is  a  necessity  for  the  evolution  of  truly  intelligent  systems. 

Architectures  must  handle  distributed  intelligence  with  flexible  control  exercised  where 
appropriate. 

Of  particular  importance  is  the  design  of  interfaces  so  the  paradigms  can  exchange  information 
and  data.  This  is  a  particularly  troublesome  problem  where  the  two  interacting  paradigms  use 
symbolic  and  numeric  data  vectors  (such  as  an  expert  system  and  a  neural  network). 


4.4  Development  Methodologies.  Testing  and  Standards 

There  is  a  considerable  lag  between  the  evolution  of  usable  software  paradigms  and  the 
generation  of  standards  for  project  development,  management  recording  and  testing,  etc.  It  was 
many  decades  before  algorithmic  software  to  control  mission  critical  activities  was  widely 
accepted.  Exploiting  the  capabilities  of  AI  paradigms  and  intelligence  distributed  both 
physically  and  across  different  paradigms  is  clearly  a  current  challenge. 

There  is  a  need  for  the  equivalent  of  DOD-STD-2167A  and  the  various  ISO  standards  [18]  so 
that  such  systems  can  be  developed,  documented,  tested  and  maintained  according  to  a  well 
understood  set  of  procedures.  Those  who  would  support  the  utilization  of  new  technologies 
would  do  well  to  support  the  development  of  such  standards. 
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4.5  At  the  LSensor  Level 

It  is  clear  that  the  continuation  of  the  development  of  smart  sensors  will  encapsulate  many  of  the 
low  level  signal  and  fusion  processing  algorithms  in  on-board  algorithms. 

In  addition  it  is  anticipated  that  better  signal-to-noise  levels  will  be  achievable  with  the  built-in 
ability  for  self  testing  and  calibration. 

Low-cost  integrated  sensors  will  enable  highly  redundant  sensors  (ten  times  the  minimal 
necessary)  to  be  a  routine  part  of  a  sensor  array. 


5.0  CONCLUSIONS 

5.1  The  Size  of  the  G&C  Problem 

It  has  been  estimated  by  Waltz  and  Buede  [17]  that  the  average  information  requirements  for 
command  and  control  of  tactical  air  warfare  is  between  25-50  decisions/minute  based  on  50,000- 
100,000  reports  from  156  separate  sensors  platforms  concerning  as  many  as  1000  hostile  targets 
being  tracked  up  to  an  altitude  of  20  km  over  an  area  of  800  square  km. 

Estimates  such  as  this  suggest  a  massive  computing  requirement,  and  certainly,  over  the  last  few 
decades,  computational  limitations  have  been  a  serious  consideration  when  considering  G&C 
systems.  Technology  has  now  provided  both  memory  and  processing  components  -  at 
reasonable  costs  -  which  to  a  large  extent  have  moved  the  problem  into  the  software  area.  The 
challenge  is  to  utilize  and  exploit  the  capabilities  of  silicon  technology  by  innovative 
architectures  and  software  paradigms. 


5.2  Distributed.  Hybrid.  Intelligent  Architectures 

G&C  systems  are  and  will  become  even  more  dependent  on  distributed  hybrid  intelligent 
architectures.  Various  paradigms  will  be  instantiated  at  the  appropriate  places  in  the  processing 
hierarchy  according  to.  functionality  demands  and  capability.  There  are  still  major  research 
problems  in  paradigm  identification  and  selection,  inter-paradigm  communications  and  in  the 
allocation  of  control. 


5.3  Methodologies.  Testing  and  Standards 

There  is  a  clear  need  for  recognized  methodologies  for  the  conduct  of  G&C  systems  that  are 
distributed  hybrid  and  intelligent.  Testing  of  expert  systems  alone  is  a  difficult  problem  without 
the  mixture  of  different  paradigms.  In  the  end  rigid  standards  (a  challenge  to  the  International 
Standards  Organization)  should  be  in  place  so  that  confidence  can  be  placed  in  the  resulting 
system. 
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5.4  A  Final  Word  on  Research  Directions 

It  is  important  to  recognize  the  requirements  for  guidance  and  control  in  military  applications  is 
considerably  different  from  non  military  applications.  The  key  difference  is  the  necessary 
assumption  that  the  game  is  being  played  against  a  malevolent  nature  (i.e.,  the  enemy)  as  distinct 
from  a  disinterested  nature  as  in  most  commercial  applications.  This  assumption  converts  the 
military  guidance  and  control  problem  into  a  zero-sum  game.  Game  theory  is  based  on  the 
assumption  that  at  each  move  in  the  game  each  player  selects  a  strategy  to  respond  to  the  last 
move  of  the  adversary;  and  associated  with  each  move  is  a  potential  pay-off.  In  a  zero-sum  game 
the  gain  of  one  player  is  the  loss  of  the  other. 

In  a  military  situation,  determining  the  world  state  vector  and  formulating  a  response  must  be 
executed  on  the  bases  that  the  world  and  possibly  the  sensors  creating  the  state  vector  are  being 
manipulated  to  maximize  the  system  losses.  The  hope  is  that  by  the  use  of  the  output  of 
multisensors  that,  in  combination,  the  information  can  be  integrated  to  provide  an  accurate 
representation  of  the  state  vector,  whereas  a  single  sensor  could  be  fooled.  Working  against  a 
hostile  player,  increases  the  demands  on  the  intelligence  required  of  the  sensor  fusion  processing. 

Military  guidance  and  control  systems  will  benefit  directly  from  the  advances  experienced  by  all 
forms  of  civilian  (industrial)  advances  in  hardware  and  software.  However,  there  is  one  area  that 
is  in  general  the  concern  only  of  military  systems  and  that  is  the  impact  on  sensor  processing 
algorithms,  data  fusion  and  integration  techniques,  and  most  importantly  the  interpretation  of  the 
world  state  vector  based  on  the  assumption  of  an  external  malevolent  manipulation  of  the  world. 
It  is  suggested  that  research  in  the  creation  of  intelligent  guidance  and  control  systems  must 
continue  to  concentrate  on  this  aspect  of  the  overall  problem. 
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a:  Centralized  Control 


b:  Decentralized  Control 

Figure  1:  General  Logical  Model  of  an  Autonomous  Robot 
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Figure  2:  Functional  Decomposition  of  Intelligent  Capabilities 


Figure  3:  A  View  of  Artificial  Intelligence  Paradigms 
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Figure  4:  A  Resource  Projecting  System 


Figure  5:  A  Process  Control  System 


Figure  6:  A  Hypothesis  Testing  System 
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Control  of  Remotely  Operated  Manipulation  Systems 

Paul  L.  Shattuck 
Martin  Marietta  Astronautics 
P.  O.  Box  179,  Mail  Code  S8500 
Denver,  CO  80201 

Abstract 

Robotic  systems  are  beginning  to  emerge  as  an  effective  alternative  to  humans  for  tasks  that  are 
repetitive.  The  next  generation  of  space  platforms,  such  as  the  international  space  station,  will  rely 
heavily  on  robotic  manipulators  for  external  servicing  and  maintenance.  This  will  enhance  the 
human  presence  in  space  and  free  the  crew  to  perform  those  tasks  that  only  humans  can  perform, 
thereby  increasing  the  science  return  of  the  station. 

Remotely  operated  manipulation  systems  have  been  accepted  and  widely  used  for  the  past  ten  to 
twenty  years  in  two  terrestrial  applications:  underwater  exploration,  servicing  and  repair;  and, 
handling  of  hazardous  materials  in  the  nuclear  industry.  The  capabilities  of  the  systems  have 
advanced  from  crude  hard-wired  master/slave  devices,  where  operators  are  adjacent  to  the 
manipulators,  to  sophisticated  manipulation  systems  that  can  reason  at  a  rudimentary  level,  with  the 
operators  half-way  around  the  world  using  telepresence  in  a  virtual  environment.  To  date, 
designing  for  the  harsh  environments  and  the  specialized,  highly  skilled  operators  required  for 
robotics  has  limited  applications  in  the  military  maintenance  depots  and  the  field.  Space  systems, 
and  associated  ground  laboratories,  have  advanced  the  manipulator  technology  and  human-machine 
interface  to  enable  a  host  of  new  cost-effective  terrestrial  applications  in  the  mining,  agricultural, 
depot  maintenance,  and  hazardous  waste  cleanup  fields.  Integration  and  packaging  developed  for 
space  solves  the  problems  of  robustness  and  tolerance  to  field  and  combat  conditions.  New 
self-contained  packaging  designs  provide  a  compact  system,  easily  used  and  transported. 
Human-machine  interfaces  and  advanced  control  systems  reduce  the  training  needs  and  ease  the 
turnover  of  skilled  personnel.  The  day  is  nearby  where  routine  tasks  (such  as  inspection, 
servicing,  and  maintenance  of  equipment)  will  be  performed  by  remotely  operated  robotic 
manipulators,  freeing  the  human  operator  to  do  the  diagnostics  and  analysis  that  only  they  can  do. 

This  lecture  will  describe  the  elements  of  a  remotely  operated  manipulation  system  from  a  require¬ 
ments  and  technology  standpoint.  Particular  emphasis  will  be  placed  on  the  system-level  require¬ 
ments  and  design  drivers  that  are  necessary  for  operations  with  humans  nearby,  or  manipulation  of 
delicate  objects.  Technology  advances  that  enable  new  applications  and  impediments  to  using 
robotics  will  also  be  discussed.  The  specific  elements  to  be  discussed,  and  their  function,  are: 

•  Actuators  and  Effectors  -  ability  to  control  and  manipulate  objects  in  the  environment; 

•  Sensing  and  Perception  -  the  machine's  interface  to  the  environment;  provides  information  to 
the  software  and  operator; 

•  Data  Management  and  Processing  -  the  computer  architecture  (processors  and  data  buses  that 
house  the  real-time  software; 

•  Control  and  Command  Execution  -  actual  software  algorithms  that  control  the  system  and 
execute  the  task  sequences;  responds  to  operator  inputs  or  sensing  and  perception  data; 

•  Operator  Interface  -  the  workstation  that  enables  the  operator  to  interact  with  the  manipulator; 
contains  the  hand  controllers,  displays,  and  computers; 

•  Task  Planning  and  Reasoning  -  the  software  that  enables  the  operator  to  script  out  and  analyze 
situations;  identifies  the  steps  required  to  operate  safely  and  accomplish  the  mission. 

A  dexterous  space  manipulator  will  be  used  to  provide  illustrative  examples  throughout  the  lecture. 
The  manipulator  is  initially  intended  to  provide  the  astronauts  a  tool  to  perform  routine  tasks  in  the 
shuttle  bay  such  as  inspection  of  items  in  the  bay  or  worksite  prepar-ation/teardown,  thereby 
improving  the  astronauts  efficiency  during  EVAs.  Control  techniques  and  the  operator  interface 
developed  for  this  system  are  planned  to  be  commercially  demonstrated  in  an  underwater  servicing 
exercise — providing  the  link  to  the  remotely  operated  manipulation  systems  that  are  the  subject  of 
this  lecture. 


Presented  at  an  AGARD  Lecture  Series  on  ‘Advanced  Guidance  and  Control  Aspects  in  Robotics’,  June  1994. 
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SUMMARY 

Robotic  systems  are  beginning  to  emerge  as  an 
effective  alternative  to  humans  for  tasks  that  are 
either  hazardous  or  routine  and  repetitive. 
Remotely  operated  manipulation  systems  have 
been  accepted  and  widely  used  for  the  past  ten 
to  twenty  years  in  two  terrestrial  applications: 
underwater  exploration,  servicing  and  repair; 
and,  handling  of  hazardous  materials  in  the 
nuclear  industry.  Technology  advances  have 
enabled  new,  broad  space  and  terrestrial  appli¬ 
cations  for  remotely  operated  manipulation 
systems.  However,  in  order  to  promulgate 
robotic  systems  further  in  the  terrestrial  work 
place,  elements  of  these  systems  must  have 
requirements  traceable  to  operations  with 
humans  nearby,  or  manipulation  of  delicate 
objects. 

A  representative  space-system  manipulator  will 
be  used  to  provide  an  illustrative  example  of  the 
evolution  of  remotely  operated  manipulation 
systems.  The  manipulator  is  initially  intended 
to  provide  the  astronauts  a  tool  to  perform 
routine  tasks  such  as  inspection,  worksite 
preparation/teardown,  or  routine  servicing  and 
maintenance,  thereby  improving  the  astronauts 
efficiency  during  EVAs. 

Emphasis  is  placed  on  the  system-level  require¬ 
ments  and  design  drivers.  Technology 
advances  that  enable  new  applications,  and 
impediments  to  using  robotics,  will  also  be 
discussed.  Control  techniques  and  the  operator 
interface  developed  for  this  manipulator  could 
be  commercially  demonstrated  in  an  underwater 
servicing  exercise  -  providing  the  link  to  the 
remotely  operated  manipulation  systems  that 
are  the  subject  of  this  lecture. 

1.0  INTRODUCTION 

Except  for  the  six  Apollo  excursions  on  the 
moon,  all  planetary  exploration  has  been 
carried  out  by  robotic  spacecraft,  albeit  systems 
with  limited  capacity  to  act  autonomously  via 


instructions  from  mission  controllers— a  tedious 
task  for  humans.  The  next  generation  of  space 
platforms,  such  as  the  international  space 
station,  will  rely  heavily  on  robotic  manipu¬ 
lators  for  external  servicing  and  maintenance. 
These  systems  will  enhance  the  human 
presence  in  space  and  free  the  crew  to  perform 
those  tasks  that  only  humans  can  perform, 
thereby  increasing  the  science  return  of  the 
station.  Within  the  next  decade  the  develop¬ 
ment  of  robots  that  can  intelligently  traverse 
large  distances,  observe  the  terrain,  manipulate 
and  analyze  samples,  service  and  maintain 
equipment,  and  report  findings  back  will  be 
feasible. 

Currently,  remotely  controlled  (teleoperated) 
and  semi- autonomous  systems  with  manipula¬ 
tion  and  mobility  exist  in  laboratories  and,  to  a 
certain  extent,  in  industry.  The  undersea  oil 
and  nuclear  power  industries  have  been  using 
intelligent,  or  teleoperated  manipulators  for  the 
past  two  decades.  Successes  in  space  with  the 
Space  Shuttle  Remote  Manipulator  System 
(RMS)— witness  the  Hubble  repair  mission- 
have  spurred  additional  technology  develop¬ 
ment.  The  capabilities  of  the  systems  have 
advanced  from  crude  hard-wired  master/slave 
devices,  where  operators  are  adjacent  to  the 
manipulators,  to  sophisticated  manipulation 
systems  that  can  reason  at  a  rudimentary  level, 
with  the  operators  half-way  around  the  world 
using  telepresence  in  a  virtual  environment. 

Nevertheless,  designing  for  the  harsh  environ¬ 
ments  and  the  specialized,  highly  skilled 
operators  required  for  robotics  has  limited 
applications  in  space,  the  military  maintenance 
depots  and  the  field.  Space  systems,  and  asso¬ 
ciated  ground  laboratories,  have  advanced  the 
manipulator  technology  and  human-machine 
interface  to  enable  a  host  of  new  cost-effective 
terrestrial  applications  in  the  mining,  agricul¬ 
tural,  depot  maintenance,  and  hazardous  waste 
cleanup  fields.  Integration  and  packaging 
developed  for  space  solves  the  problems  of 
robustness  and  tolerance  to  field  and  combat 
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conditions.  New  self-contained  packaging 
designs  provide  a  compact  system,  easily  used 
and  transported.  Human-machine  interfaces 
and  advanced  control  systems  reduce  the 
training  needs  and  ease  the  turnover  of  skilled 
personnel.  The  day  is  nearby  where  routine 
tasks  (such  as  inspection,  servicing,  and  main¬ 
tenance  of  equipment)  will  be  performed  by 
remotely  operated  robotic  manipulators,  freeing 
the  human  operator  to  do  the  diagnostics  and 
analysis  that  only  they  can  do. 

Exhibit  1-1  provides  the  topics  for  the 
remaining  section  of  this  report.  Exhibit  1-2 
illustrates  trends  in  the  development  and 
maturity  of  remotely  operated  manipulators. 

2.0  SYSTEM  DESCRIPTION 

It  is  useful  to  decompose  a  Remotely  Operated 
Manipulation  System  into  functional  elements 
for  discussion  purposes.  For  the  purpose  of 
this  discussion  the  elements,  and  their  function, 
are: 

•  Actuators  and  Effectors  -  ability  to  control 
and  manipulate  objects  in  the  environment; 

•  Sensing  and  Perception  -  the  machine's 
interface  to  the  environment;  provides 
information  to  the  software  and  operator; 

•  Data  Management  and  Processing  -  the 
computer  architecture  (processors  and  data 
buses  that  house  the  real-time  software; 

•  Control  and  Command  Execution  -  actual 
software  algorithms  that  control  the  system 
and  execute  the  task  sequences;  responds  to 
operator  inputs  or  sensing  and  perception 
data; 

•  Operator  Interface  -  the  workstation  that 
enables  the  operator  to  interact  with  the 
manipulator;  contains  the  hand  controllers, 
displays,  and  computers; 

»  Task  Planning  and  Reasoning  -  the  soft¬ 
ware  that  enables  the  operator  to  script  out 
and  analyze  situations;  identifies  the  steps 
required  to  operate  safely  and  accomplish 
the  mission. 

These  elements  and  their  physical  implemen¬ 
tation  are  shown  in  Exhibits  2-1  and  2-2; 
internal  and  external  support  needs  are  listed  in 
Exhibit  2-3. 


One  must  first  understand  the  underlying 
requirements  that  drive  a  robotic  system.  The 
next  section,  2.1— System  Requirements  and 
Design  Criteria— will  discuss  those  require¬ 
ments,  and  the  associated  design  criteria,  which 
form  the  basis  for  a  remotely  operated  manipu¬ 
lator.  Following  that  discussion  will  be  a 
description  of  the  physical  components  that 
make  up  a  robotic  system.  System  Design 
Elements,  Section  2.2,  focuses  on  those  design 
elements  that  control  a  robotic  system.  Control 
issues  associated  with  remotely  operated 
manipulators  are  highlighted  in  Section  2.3, 
System  Control  and  the  Operator  Interface. 

2.1  System  Requirements  and  Design 
Criteria 

The  approach  used  to  develop  the  requirements 
for  a  robotic  system  is  an  iterative  one  which 
relies  on  a  definition  of  the  tasks  required  to  be 
performed  by  the  system.  Knowledge  of  the 
task  definition,  resources  available,  worksite 
constraints,  etc.,  are  essential  to  avoid  over¬ 
specification  of  requirements.  Parallel  empha¬ 
sis  needs  to  be  placed  on  the  state  of  the 
technology  base  to  ensure  that  technology 
development  and  validation  plans  are  in  place  to 
support  the  program.  As  systems  are  fielded, 
operational  lessons  learned  and  new  technology 
needs  can  be  fed  back  into  the  robotic  devel¬ 
opment  effort. 

The  optimal  definition  of  a  robotic  system 
(manipulator  kinematics,  mobility,  operator 
interface,  etc.)  stems  from  a  task  analysis  to 
avoid  specification  of  unneeded  capabilities  or 
features.  An  understanding  of  the  task  require¬ 
ments  (worksite  constraints,  task  element 
characteristics,  sequence  of  operations)  leads  to 
kinematic  and  dynamic  simulation  that  develops 
the  manipulator  configuration,  vision  system 
requirements,  and  collision  avoidance  require¬ 
ments.  Additionally,  the  robotic  system  duty 
cycles,  which  drive  the  host  platform  resource 
requirements,  are  directly  related  to  the  task 
sequence  of  events.  Exhibit  2-4  portrays  the 
task-driven  methodology. 

It  is  critical  to  establish  a  complete  set  of  detail¬ 
ed  system  and  lower-level  specifications  early 
in  the  program,  well  before  hardware  design  is 
initiated.  Realistic  margins  must  be  established 
in  the  requirements,  to  allow  for  unplanned 
contingencies  as  the  design  progresses.  Of 
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equal  importance  at  this  time  in  the  program  is 
to  plan  ahead  for  verification  of  the  require¬ 
ments— all  requirements  should  be  established 
in  the  form  of  quantifiable,  verifiable  specifics. 
The  appropriate  level  of  requirements  trace- 
ability  must  be  clearly  established,  with 
concurrence  from  the  developing  agency  and 
user  communities. 

Establishing  the  overall  system  performance, 
based  on  a  specific  mission  simulation,  is 
critical  in  establishing  requirements  flow-down 
and  allocation  to  lower-level  subsystems  and 
hardware.  Key  performance  requirements  are 
listed  in  Exhibit  2-5.  For  a  robot,  the  manipu¬ 
lator  and  its  required  performance,  are  the  key 
to  ensuring  mission  success.  Realistic  force/ 
torque  requirements  at  the  interface  between  the 
manipulator  and  the  work  piece  need  to  be  set 
to  enable  the  manipulator  to  perform  all  its 
assigned  tasks.  The  kinematics  of  the  manipu¬ 
lator  and  the  relationship  with  the  worksite 
must  be  considered,  with  respect  to  force/ 
torque  and  also  to  power.  Realistic  margins  on 
both  minimum  and  maximum  performance 
should  be  set  up  that  will  allow  the  manipulator 
to  perform  its  assigned  tasks,  but  yet  consider 
power  and  safety  implications.  While 
minimum  requirements  ensure  that  the 
manipulator  can  perform  its  tasks,  maximum 
requirements  help  limit  loads  on  the  manipula¬ 
tor  and  the  end  effector,  and  on  all  work  pieces 
with  which  it  comes  in  contact.  In  addition  to 
placing  requirements  on  static  forces  and 
torques,  considerations  should  also  be  given  to 
specifying  joint  and  Cartesian  velocities  and 
accelerations.  Exhibit  2.6  summarizes  the 
considerations  for  establishing  system-level 
performance  force/torque  requirements. 

One  of  the  key  drivers  affecting  the  design  of 
remotely  operated  manipulators,  when  a  human 
or  fragile/expensive  equipment  is  nearby,  is  the 
safety  requirements.  Exhibit  2-7  provides  a 
correlation  of  the  safety  requirement  for  a  dex¬ 
terous  space  manipulator  and  resultant  design 
impact.  Safety  needs  to  be  addressed  early  in 
the  development  cycle  as  it  has  major  impacts 
on  the  system  architecture  and  associated  hard¬ 
ware  and  software  design.  These  are  among 
the  first  set  of  requirements  that  must  be 
established  and  frozen.  Past  robotic  systems  in 
terrestrial  and  space  applications  have  suffered 
from  the  lack  of  understanding  of  the  safety 


issues.  After  the  system  is  built  and  fielded, 
the  only  recourse  to  a  safety  inadequacy  is  to 
limit  the  operations  of  the  manipulator,  and 
hence  its  utility. 

Some  of  the  important  worksite  interface  con¬ 
siderations  used  in  manipulator  design  are 
discussed  in  Exhibit  2-8.  All  of  these  items 
should  be  considered  very  early  in  the  design, 
and  re-evaluated  for  impacts  whenever  design 
changes  are  considered.  The  kinematic  analy¬ 
ses  are  performed  on  a  simulator  model  that 
later  can  be  integrated  into  the  operator 
workstation  for  task  planning  and  preview. 
This  model  is  used  to  evaluate  manipulator 
dexterity,  viewing  requirements,  work  piece 
locations,  and  viewing/guiding  aids.  In  addi¬ 
tion  to  the  computer  simulations,  lab  tests  using 
realistic  hardware  also  are  essential  to  help 
establish  performance  requirements  and  to 
evaluate  the  design.  The  criticality  of  the  usage 
of  these  tools  in  establishing  and  optimizing 
human-machine  interfaces,  and  in  establishing 
and  evaluating  detailed  requirements  and 
performance  capabilities  cannot  be  over-empha¬ 
sized. 

When  designing  elements  for  robotic  servicing, 
one  must  keep  in  mind  that  all  manipulators 
have  a  preferred  work  area  which  maximizes 
their  dexterity.  Dexterity  requirements  are 
dependent  on  the  tasks  performed.  A  single 
grasp  point,  with  a  nut  runner  interface  and 
straight  line  removal  is  preferable  from  a 
robotics  standpoint.  Dexterity  is  then  directly 
proportional  to  the  removal  distance.  Another 
constraint  placed  on  a  designer  by  robot 
servicing  is  clearance  envelopes.  There  is  a 
minimum  diameter  about  the  grasp  point 
required  by  EVA  standards.  For  robotic  servi¬ 
cing  one  must  also  consider  visual  clearance 
envelope  to  allow  the  camera  to  "see"  the 
alignment  cues.  Exhibit  2-9  illustrates  design 
concepts  for  robotic  compatibility. 

Human-machine  interface  considerations 
should  be  used  extensively  throughout  the 
system  development.  They  are  used  to  evaluate 
the  interaction  between  the  human  operator  and 
environment  and  in  layout  and  operation  of  the 
workstation.  A  key  tool  is  a  prototype  simula¬ 
tor  to  develop  and  evaluate  the  hardware  and 
software  required  for  displays  and  controls. 
Critical  to  the  design  and  acceptance  of  robots 
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is  the  early  involvement  of  the  end  users.  This 
permits  that  lessons  learned  from  previous 
experience  can  be  input  early  in  the  design, 
gives  the  system  designers  and  system  users 
the  chance  to  interact,  and  ensures  vital  support 
from  the  people  that  will  use  the  hardware  in 
operations.  Key  considerations  are  reiterated  in 
Exhibit  2-10. 

Do  not  forget  to  determine  the  data  the  opera¬ 
tors  and  logistics  support  team  need  to  check 
out  the  system,  determine  the  performance  of 
the  system,  and  diagnose  anomalies.  Date  con¬ 
tent  and  rate  can  impact  the  software  design  and 
processor  architecture.  Refer  to  Exhibit  2- 1 1 
for  a  data  checklist. 

2.2  System  Design  Elements 

The  functional  elements  of  any  remotely 
operated  manipulation  systems  were  defined  in 
Section  2.0,  with  an  illustrative  physical  mani¬ 
festation  portrayed  in  Exhibit  2-2  (a  dexterous 
space  manipulator).  Focus  in  this  section  will 
be  on  the  three  main  components  of  the  system, 
which  embody  all  six  functions:  1)  the 
Manipulator,  2)  the  Command  and  Data 
Processing  System,  and  3)  the  Operator  Work¬ 
station. 

An  example  of  a  dexterous  manipulator  is 
shown  in  Exhibit  2-12.  Key  characteristics  of 
the  manipulator  are  listed  in  Exhibit  2-13.  In 
this  example,  a  portion  of  the  Command  and 
Data  Processing  System  is  imbedded  in  the  arm 
links.  This  manipulator  is  a  robotic  arm  with 
seven  degrees  of  freedom.  The  joints  are 
configured  to  provide  a  closed-form  inverse 
kinematic  solution  and  in  such  a  way  that  the 
manipulator  has  singularities  only  at  limits  of 
travel.  The  manipulator  is  comprised  of  several 
smaller  subsystems:  actuators,  force  torque 
transducer,  end-of-arm  tooling,  vision  system, 
and  electronic  controllers.  All  structural  mem¬ 
bers  are  designed  to  provide  required  stiffness 
to  achieve  minimum  bandwidth  frequency 
under  active  control.  It  is  designed  to  be 
reconfigurable  to  either  a  right-  or  left-handed 
configuration  on  adjustment  of  the  shoulder 
yaw  hardstops.  Key  performance  requirements 
are  defined  as  forces,  torques,  and  speeds  that 
the  end  of  the  arm  must  achieve  under  worst 
case  conditions. 


The  manipulator  is  powered  by  regulated  120 
volts  and  controlled  through  distributed  elec¬ 
tronics  located  in  the  shoulder,  upper  arm,  and 
lower  arm  links.  A  1553  data  bus  and  video 
signals  are  routed  throughout  the  arm  and  up  to 
the  tool  plate.  An  electrical  secondary  path  is 
provided  to  operate  the  manipulator  in  a 
degraded  mode  as  required  to  complete  a 
mission  or  store  the  arm  in  case  of  a  failure  in 
the  primary  control  system.  A  thermal  control 
system  is  utilized  to  maintain  acceptable  elec¬ 
tronic  part  temperatures  in  the  controllers  and 
actuators.  The  actuators  have  a  manual  release 
mechanism  that  allows  a  human  to  release  a 
jammed  actuator  and  manually  backdrive  a  joint 
for  stowage. 

The  manipulator  joint  actuator  provides  the 
positioning  capability  for  the  manipulator  and 
the  torque  required  to  generate  forces  and 
torques  at  the  manipulator  tool  plate.  The 
actuators  are  designed  to  produce  the  required 
manipulator  performance  in  the  primary  control 
operations  mode,  as  well  as  degraded  perfor¬ 
mance  in  a  secondary  mode.  They  also 
electrically  link  the  distributed  controllers  and 
other  subsystems  together  along  the  manipu¬ 
lator  arm. 

Each  actuator  typically  includes  a  motor, 
transmission,  output  position  sensor,  output 
torque  sensor,  fail-safe  input  brake,  hardstops, 
temperature  sensors,  and  a  flexible  cable  wrap. 
The  shoulder  roll  actuator  may  also  have  a 
brake  at  its  output.  Elousings  are  designed  to 
provide  required  stiffness  to  achieve  the 
minimum  bandwidth  frequency  of  the  manipu¬ 
lator  under  active  control. 

The  conductor  cable  allows  power,  signal  and 
balanced  lines  to  be  routed  to  and  through  each 
actuator  while  allowing  freedom  of  motion  and 
minimum  disturbance  torque.  Each  cable  is 
uniquely  configured  to  fit  within  an  actuator 
and  allow  proper  joint  rotation  while  providing 
static  interface  connections  between  actuators 
or  controllers. 

A  sensor  measures  the  joint  position.  This 
joint  position  is  then  supplied  to  the  data 
management  and  processing  system  as  inputs 
to  the  control  system  and  for  safety  checks  to 
prevent  a  "runaway"  manipulator. 
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Vision  and  tactile  sensing  are  the  major  senses 
involved  in  manipulation.  Examples  of 
different  alternatives  to  provide  these  sensors  is 
contained  in  Exhibit  2-14. 

The  wrist  camera  provides  a  close-up  view  of 
the  task  being  performed,  and  typically  the 
field-of-view  includes  a  portion  of  the  end 
effector  for  relative  orientation.  Lighting  can 
be  included  to  provide  uniform  illumination  of 
the  task  area-a  requirement  for  some  image 
processing  systems. 

The  Force/Torque  Transducer  at  the  end  effec¬ 
tor  interface  to  the  arm  provides  data  in  six 
degrees  of  freedom  (three  forces,  three 
moments)  when  in  contact.  This  data  is  used 
for  control  of  the  manipulator  as  well  as  safety 
for  prevention  of  excessive  force. 

There  are  two  distinct  means  to  use  the  force 
and  torque  data  for  control.  In  the  first  case 
data  can  be  provided  to  the  operator  at  a 
workstation  equipped  with  a  force-reflected 
hand  controller.  This  is  most  useful  in  tele¬ 
operation  where  the  operator  physically  "feels" 
the  force  exerted  on  the  worksite.  The 
alternative  to  this  approach  is  to  use  limits  set  in 
software  to  autonomously  adjust  the  amount  of 
force  applied  to  the  worksite.  This  is  a  type  of 
control  known  as  compliant  control  or  force- 
moment  accommodation.  Visual  aids  and 
graphics  can  be  supplied  to  the  operator  for 
supervisory  tasks. 

2.3  System  Control  and  the  Operator 
Interface 

Robots  with  multiple  degree-of-freedom  (DOF) 
manipulators  are  very  complex  machines  to 
control.  There  is  constant  concern  with  the 
location  of  objects  in  three-dimensional  space. 
These  objects  are  the  manipulator  links,  the 
end-effectors  and  tools,  and  elements  of  the 
work  space.  Location  of  these  objects  is 
described  by  their  position  and  their  orien¬ 
tation.  The  goal  of  manipulator  control  is  to 
place  the  end-effector  at  the  position  of  the 
work  piece  to  be  manipulated  with  a  relative 
orientation  that  will  provide  the  required  dex¬ 
terity  to  accomplish  the  task. 

A  few  definitions,  listed  in  Exhibit  2-15,  are  in 
order  before  proceeding  further  in  the 
discussion  of  control.  Kinematics  is  the 


science  of  motion  without  regard  to  the  forces 
which  cause  the  motion.  The  existence  of  a 
kinematic  solution  defines  the  work  space  of 
the  manipulator.  The  manipulator  inverse 
kinematics,  a  nonlinear  problem,  are  the  key  to 
position  control  of  the  end-effector. 

The  manipulator  is  a  series  of  nearly  rigid  links 
which  are  connected  by  joints  to  allow  relative 
link  motion.  The  joints  are  either  revolute 
(rotary)  or  prismatic  (extending).  These  links 
and  joints  form  a  kinematic  chain  with  the  free 
end  being  the  end-effector.  When  in  contact 
and  grasping  the  work  piece,  the  open 
kinematic  chain  becomes  closed,  and  the 
dynamic  equations  governing  control  are 
altered  -  increasing  the  complexity  of  control. 

With  this  background  in  hand,  there  are  several 
major  trades  that  shape  the  control  system 
design;  these  trades  are  summarized  in  Exhibit 
2-16.  The  two  ends  of  the  spectrum  for  oper¬ 
ator  control  are:  1)  Teleoperated,  or  guided  by 
a  human  on  a  continual  basis  at  low  or  high 
levels  and  from  some  distance  with  possible 
time  lag;  and  2)  Supervisory  Control  where 
automated  robots  carry  out  a  specified  set  of 
pre-programmed  functions  and  robots  with  a 
higher  degree  of  autonomy  actually  respond  to 
new  situations  with  little  or  no  additional 
guidance  from  the  operator. 

Key  to  the  successful  completion  of  any 
teleoperated  robotic  task  is  the  environment 
with  which  the  robot  needs  to  interact.  A  well- 
defined  environment  (minimum  clutter  and 
precisely  defined  in  the  model  of  the  world) 
reduces  the  amount  of  intelligence  required  to 
accomplish  the  task.  In  a  teleoperated  scenario, 
the  operator  is  in  the  control  loop  and  uses  his 
own  senses/intelligence  to  understand  the 
environment,  manipulate  objects,  and  close  the 
feedback  loop. 

Under  supervisory  control,  the  operator  must 
divide  time  between  monitoring  the  systems 
performance  and  giving  commands  to  the 
computer.  Monitoring  is  a  stochastic  process 
and  can  only  predict  that  an  event  will  occur, 
but  cannot  predict  the  time  or  place.  This 
stochastic  nature  forces  the  operator  to  maintain 
a  constant  vigilance  on  the  system's  operations. 
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The  location  of  the  operator  relative  to  the  robot 
is  an  important  aspect  of  any  robotic  system 
architecture.  When  the  operator  and  robot  are 
separated  by  a  large  distance  (e.g.,  robot  in  low 
earth  orbit  and  operator  on  the  ground), 
communication  delays  can  be  a  significant 
factor  in  system  design.  A  large  delay  implies 
that  special  compensating  measures  such  as 
increased  automation,  predictive  displays,  or 
sensor-based  collision  avoidance  need  to  be 
incorporated.  On  the  other  hand,  small 
distances  between  robot  and  operator  (e.g., 
operator  and  robot  located  contiguously), 
implies  increased  utility  of  teleoperation  and 
capabilities  such  as  force  reflection. 

Furthermore,  the  separation  distance  between 
robot  and  operator  affects  contingency  planning 
and  task  viewing.  When  the  operator  is 
collocated  with  the  robot,  it  may  mean  that 
human  operations  are  possible  to  assure  task 
performance  under  contingencies.  When 
human  backup  is  not  possible,  extra  redun¬ 
dancy  and  operational  margins  may  be 
necessary  in  the  robot  system  to  achieve  the 
desired  probability  of  success  for  critical  tasks. 
Also,  when  the  operator  location  permits  direct 
viewing  of  the  robot  task,  there  is  a  reduced 
need  for  camera  views  and  workstation 
monitors. 

Once  a  decision  is  reached  on  the  degree  of 
operator  involvement,  a  partitioning  of  func¬ 
tions  between  the  "local”  operator  site  and 
"remote"  robotic  worksite  is  defined.  The 
driver  to  partitioning  functions  between  local 
and  remote  sites  is  to  make  the  system  less 
sensitive  to  issues  such  as  intermittent 
communications,  time  delays,  and  communi¬ 
cation  bus  latencies-all  of  which  tend  to  be 
destabilizing.  A  sample  partitioning  of  func¬ 
tions  is  shown  in  Exhibit  2-17. 

The  controls  algorithms  provide  the  means  for 
the  robotic  hardware  to  accomplish  its  intended 
performance.  A  sample  dexterous  space 
manipulator  control  architecture  is  depicted  in 
Exhibit  2-18.  There  is  an  "inner"  torque  loop 
that  precludes  the  application  of  excessive  force 
at  the  worksite.  A  Cartesian  position  loop  then 
compares  the  sensed  actual  position  to  the 
commanded  position  and  orders  appropriate 
motion.  Finally,  the  outermost  loop  provides 
force  feedback  to  the  operator  so  that 


teleoperation  can  be  accomplished  with  a  sense 
of  "feel."  This  promotes  dexterous  operations 
required  for  fine  manipulation  of  objects.  The 
partitioning  of  functions  between  the  operator 
workstation  (local)  and  manipulator  (remote)  is 
further  magnified  in  Exhibit  2-19. 

A  key  feature  of  the  controls  algorithms 
includes  6-DOF  active  control  with  the  seventh 
shoulder  roll  joint  operated  in  an  indexing 
mode  to  control  the  placement  of  the  elbow 
(i.e.,  elbow  up,  elbow  down,  or  selectable 
anywhere  in  between).  The  seventh  joint  could 
be  included  actively  in  the  control  algorithm  to 
form  a  redundant  control  architecture.  The 
controls  algorithms  permit  the  system  to  be 
operated  via  teleoperation,  or  in  an  autonomous 
mode,  with  selectable  levels  of  force  reflection 
available  in  the  teleoperated  mode,  and  select¬ 
able  levels  of  impedance  control  available  in 
both  teleoperations  and  autonomous  control. 
The  functional  requirements  for  the  control 
system  are  listed  in  Exhibit  2-20.  Some  of  the 
key  controls  issues  include  contact  stability, 
contact  force  spikes,  impedance  control,  inertia 
decoupling  and  force  reflection. 

The  control  algorithms  must  be  specified  using 
a  standard.  One  option  for  guidance  is  the 
NASA/NIST  Standard  Reference  Model  for 
Robotic  Systems  (NASREM),  portrayed  in 
Exhibit  2-21.  The  control  algorithms  discussed 
previously  concentrated  on  the  servo  and 
primitive  levels. 

The  servo  level  incorporates  all  real-time 
processing  required  to  meet  the  control  loop 
timing  requirements.  The  servo  level  is  sub¬ 
divided  into  workstation  and  telerobot 
algorithms.  The  data  interfaces  for  the  servo 
level  can  be  characterized  as  follows: 

•  Hand  Controller  to  Workstation  Algorithms 
—  sensed  hand  controller  joint  positions; 

•  Workstation  to  Telerobot  Algorithms  — 
commanded  (teleoperated)  object  velo¬ 
cities; 

•  Telerobot  Algorithms  to  Telerobot  Manipu¬ 
lator  —  commanded  actuator  currents; 

•  Telerobot  Manipulator  to  Telerobot  Algo¬ 
rithms  —  sensed  tool  plate  forces  and 
torques; 


4-8 


•  Telerobot  to  Workstation  Algorithms  — 
sensed  (teleoperated)  object  forces  and 
torques; 

•  Workstation  Algorithms  to  Hand  Controller 
—  commanded  actuator  currents. 

The  primitive  level  incorporates  all  processing 
required  to  define,  execute  and  monitor  path 
planning  between  way  points  in  either 
Cartesian  or  joint  space.  The  primitive  level 
passes  position  commands  (either  Cartesian  or 
joint)  to  the  servo  level  at  the  servo  level 
control  rate,  and  monitors  exception  conditions 
to  define  error  conditions. 

The  operator  workstation  is  the  human-machine 
interface  for  the  robotic  system.  For  a  given 
robotic  system,  the  only  difference  between 
ground-based  workstations  is  the  operator's 
"chair."  The  common  interface  devices  provide 
input,  feedback,  and  monitor  functions.  The 
balance  between  teleoperation  and  autonomy 
for  the  specific  robotic  system  determines  the 
character  and  requirements  of  these  interface 
devices.  The  very  nature  of  teleoperation 
requires  hand  controllers  as  input  devices.  A 
fully  autonomous  system  might  use  only  a 
keyboard  and  a  mouse  for  input  devices.  The 
amount  and  form  of  feedback  from  the  sensors 
in  the  system  is  a  function  of  the  types  of  tasks 
to  be  performed  and  system  considerations. 
For  example,  force  feedback  to  the  hand  con¬ 
troller  aids  teleoperation  during  fine  dexterous 
manipulation. 

In  order  for  the  robot  operator  to  monitor  task 
performance,  the  state  of  the  work  space,  and 
the  state  of  the  robot,  data  must  be  transmitted 
from  the  robot  to  the  operator  workstation. 
Likewise,  data  transfer  from  the  workstation  to 
the  robot  is  necessary  to  allow  the  operator  to 
issue  commands  that  control  robot  functions. 
Data  exchange  is  accomplished  in  one  of  two 
ways:  hardwire  connection  or  RF  link  between 
robot  and  operator.  Hardwire  data  exchange, 
which  includes  fiber  optic  links,  has  the 
advantage  of  high  bandwidth  that  readily 
accommodates  transmission  of  multiple  video 
channels  and  force  feedback  data.  RF  links 
allow  data  exchange  over  large  distances  such 
as  low  earth  orbit  to  ground,  but  are  often 
bandwidth-limited. 


Lastly,  to  accomplish  the  task  efficiently  not 
only  requires  the  right  robotic  system  for  the 
task,  but  requires  a  fully  trained  operator  well 
versed  in  the  system  and  task  at  hand. 
Simulators  and  kinematically  correct  trainers 
can  reduce  task  times  by  a  factor  of  1.5  to  3, 
which  is  significant  when  performing  servicing 
and  maintenance  operations. 

3.0  APPLICATIONS 

Extending  the  useful  life  of  operational  systems 
via  maintenance,  assembly,  and  repair  requires 
dexterous  manipulation.  Future  proposed 
missions  envision  a  dexterous  manipulation 
environment  that  is  more  hazardous,  remote, 
and  extensive  than  the  current  mission 
scenarios.  In  this  hostile  environment  an 
intelligent  system  that  is  capable  of  dexterous 
manipulation  would  enhance  or,  in  some  cases, 
be  critical  to  mission  success.  Examples  of 
potential  applications  are  contained  in  Exhibit 
3-1. 

The  following  section,  3.1,  discusses  some 
potential  uses  for  remotely  operated  manipu¬ 
lators.  In  Section  3.2  a  questionnaire  is 
provided  to  help  understand  issues  associated 
with  selecting  robotic  systems. 

3.1  Current  and  Future  Uses 
Teleoperated  robots  are  designed  to  extend 
human's  presence  in  hazardous  environments 
such  as  nuclear  radiation,  high  temperature 
applications  such  as  burning  buildings, 
undersea  operations  at  tremendously  high 
pressure  (extreme  depths),  and  the  harsh  space 
environment.  These  environments  limit 
human's  performance  and  can  endanger  life. 
Teleoperated  systems  are  implemented  because 
of  their  lower  cost  and  maturity  in  fielding  a 
working  system.  Ease  of  implementation  to  an 
untrained  operator  reduces  cost  and  can  be  a 
time  saving  advantage  to  everyday  operations. 
There  are  some  current  successes  using 
remotely  operated  systems. 

The  nuclear  power  industry  has  made 
significant  use  of  glove  boxes,  manipulators 
and  mobile  robots  for  work  in  high-radiation 
environments.  In  the  mid-1980s,  cleanup  of 
the  Three-mile  Island  power  plant  in 
Pennsylvania  was  accomplished  with  mobile, 
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teleoperated  robots  equipped  with  tools  for 
inspection,  cutting,  drilling,  and  welding. 

The  undersea  oil  industry  has  used  ROVs 
equipped  with  manipulators  to  inspect  oil  rigs 
on  the  sea  floor  and  pipelines  for  weld 
integrity.  The  National  Science  Foundation,  in 
conjunction  with  NASA,  is  furthering  undersea 
use  with  an  advanced  human-machine  interface 
—  Telepresence  —  in  Antarctica. 

In  industry,  a  tiny  constrained  niche  exists  for 
robots  performing  fix-based  manipulation  in 
highly  structured  tasks  with  only  operator 
supervision.  This  is  used  extensively  in 
pharmaceutical  and  assembly-line  manufac¬ 
turing. 

Broader  use  of  robotics  is  hampered  in  many 
areas  because  of  technology  or  system 
limitations.  Many  of  these  limita-tions  are 
being  eliminated  by  new  ways  of  integrating 
and  packaging  robotics. 

Reliability,  fault  tolerance,  and  safety  features 
of  space  manipulators  can  be  transferred  to 
increase  the  use  of  robotics  in  handling, 
manufacturing,  and  testing  of  extremely  high- 
value  hardware.  There  are  many  areas  that 
would  see  increased  profits  and  availability  if 
they  could  be  built  using  batch  processes, 
eliminating  expensive  and  slow  custom 
building.  The  totally  integrated  operational 
hierarchical  control  system  discussed  earlier 
will  pave  the  way  for  advanced  path  and  task 
planning,  thus  allowing  cost-effective  use  of 
robotics  in  low-volume  manufacturing. 

Robotics  in  hazardous  areas  such  as  nuclear 
reactor  maintenance  need  to  be  teleoperated. 
Teleoperated  robots  lower  the  risk  to  people 
and  allow  more  efficient  operations  or  clean¬ 
ups.  Higher  levels  of  control  allow  pure 
teleoperation,  supervised  autonomy,  or  com¬ 
pletely  automated  control.  These  features 
reduce  the  need  for  highly  skilled  operators  and 
reduce  operator  fatigue.  Increased  effec¬ 
tiveness  and  better  use  of  our  small  population 
of  highly  skilled  operators  make  robotic 
benefits  vital  in  the  areas  of  fire  fighting, 
asbestos  removal,  pollution  cleanup,  and  other 
hazards. 


The  harsh  environments  and  specialized, 
skilled  operators  required  for  robotics  in 
maintenance  depots  or  the  field  have  until  now 
limited  robotics  in  the  military.  Integration  and 
packaging  developed  for  space  solves  the 
problems  of  robustness  and  tolerance  to  field 
and  combat  conditions  that  had  limited  robotics 
use  by  our  armed  forces.  The  self-contained 
packaging  concept  provides  a  compact  system, 
easily  used  and  transported.  New  human- 
machine  interfaces  and  advanced  control 
system  reduce  the  training  needed  and  eases  the 
turnover  of  skilled  personnel. 

Currently  robotics  are  extensively  used  in 
underwater  exploration  and  servicing. 
However,  much  like  hazardous  environment 
maintenance,  undersea  exploration  and  servi¬ 
cing  needs  more  than  predominantly  pure  tele¬ 
operation  offered  by  current  robotics.  Compact 
electronics  packaging  allows  extensive  onboard 
capability,  reducing  reliance  on  unwieldy 
umbilicals  and  more  flexibility  than  pure 
master-slave  control. 

Ongoing  program  development  activities  have 
direct  applicability  for  lunar  and  planetary 
exploration.  Use  of  already  space-qualified 
robotic  components  would  reduce  the  costs  for 
future  systems.  The  supervised  autonomy  and 
high-level  autonomous  control  architecture  will 
allow  better  mission  control,  as  the  current  pure 
teleoperation  suffers  from  long  communication 
time  delays.  A  self-contained  compact  com¬ 
puter  architecture  provides  the  computational 
resources  required  for  these  missions. 

The  development  of  on-orbit  satellite  servicing 
has  been  shown  to  be  highly  cost  effective. 
Required  technologies  can  be  directly  trans¬ 
ferred  from  existing  research  including 
manipulators,  data  system,  control  system, 
vision,  sensors,  and  power.  The  flight-devel¬ 
opment  status  of  these  technologies  will 
significantly  shorten  the  development  time  and 
cost  of  satellite  servicing  systems. 

Current  technology  restricts  the  use  of  robotics 
in  applications  such  as  agriculture,  construc¬ 
tion,  and  warehouse  operations.  The  integra¬ 
tion  of  robotic  technologies  and  the  compact 
packaging  discussed  here  provide  a  blueprint 
for  similar  commercial  implementations.  The 
high  reliability,  safety,  extensive  integrated 
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computing  capability,  and  human-machine 
interface  features  address  the  most  critical 
obstacles  limiting  robotics  use  in  these  areas. 
Cost-effective  robotic  products  for  these  areas 
increase  efficiency  and  productivity. 

3.2  Application  Questionnaire 

The  following  series  of  questions  help  to  define 
the  selection  of  the  proper  robotic  system  for  a 
given  application. 

Understanding  the  Environment 

•  What  type  of  operations  or  activities  charac¬ 
terize  your  industry  or  field? 

-  Any  tasks  currently  performed  by 
machines  (pick  and  place,  NC,  operator 
controlled,  etc.)? 

-  Any  repetitive  tasks? 

-  Any  time-intensive  tasks? 

-  Any  tasks  hazardous  to  humans? 

•  What  characterizes  the  environment? 

-  Is  it  clean  or  dirty  (particulates,  radia¬ 
tion,  moisture,  etc.)? 

-  What  is  the  gravity  (on  Earth- lg,  in 
space— Og,  on  a  planet,  moon,  etc.)? 

-  What  is  the  pressure  (ambient,  space, 
undersea,  etc.)? 

•  What  are  the  constraints  and  limitations 
placed  on  personnel  and  machines  in  the 
work  space? 

-  Are  there  weight  limits  (floor  loads, 
transport  to/from  limits,  etc.)? 

-  Are  there  volumetric  constraints  (width/ 
height  of  corridors,  clear  volume  in 
work  cell,  etc.)? 

-  Are  there  obstructions  to  work  around? 

-  Is  there  clear  visibility  of  the  task  being 
performed? 

Understanding  the  Task 

•  What  are  the  characteristics  of  the  work 
space? 

-  Does  it  contain  rigid,  structured,  un¬ 
changing  work  cells  capable  of  being 
accurately  modeled  a  priori? 

-  Are  there  areas  that  are  unstructured  and 
changing  that  require  real-time  know¬ 
ledge  of  the  work  space? 

•  What  are  the  attributes  of  the  tasks  being 
performed? 

-  Is  contact  required  (stiff,  soft,  variable 
stiffness)? 

-  Is  precise  position  or  velocity  control 
needed? 


-  Is  base  motion  required? 

-  Is  sensory  interpretation  required 
(visual  analysis,  gas  detection,  etc.)? 

•  Does  it  require  manipulating  objects? 

-  Are  they  attached  or  in  free  space? 

-  Do  they  have  variable  mass  and  inertia; 

-  Any  offset  CGs? 

-  Is  the  grasp  point  designed  for  robotics 
(standard  interface,  special  tool,  etc.)? 

•  Are  there  safety  considerations  involved? 

-  Is  inadvertent  release  of  objects  or  tools 
a  concern  (damage  to  the  object,  dam¬ 
age  to  surroundings,  etc.)? 

-  Is  excessive  force  or  contact  a  concern 
(breakage,  deformation,  etc.)? 

-  Does  a  runaway  manipulator  cause  con¬ 
cern  (injury  to  humans,  damage  to  envi¬ 
ronment,  damage  to  manipulator,  etc.)? 

Understanding  the  Operator  Interface 

•  If  robotics  are  currently  in  use,  what  is  the 

operator  involvement? 

-  Is  it  real-time  control  of  the  device  or 
supervision? 

-  Is  the  operator  located  locally  or 
remotely? 

-  What  are  the  operator  inputs  to/from  the 
machine  (to-high-level  task  commands, 
hand  controllers,  etc.;  from-visual, 
force  feedback,  graphical,  etc.)? 

•  What  are  the  operator  desires? 

-  Do  they  want  direct  control,  super¬ 
visory  responsibilities,  or  a  mix? 

-  If  it's  a  manipulator,  do  they  want 
single  joint  control,  joysticks,  master/ 
slave,  or  some  other  kinematic  mapping 
of  commands? 

-  How  does  the  operator  envision  the 
workstation  to  look  (portable  box  with 
joystick  and  knobs,  sophisticated 
displays  and  processor  interfaces  and 
hand  controllers,  etc.)? 

-  Should  the  workstation  be  located 
nearby,  with  direct  line-of-sight? 

-  What  feedback  is  desired  (camera  views 
and  how  many,  force  feedback,  graphi¬ 
cal  overlays,  etc.)? 

•  How  will  communications  occur  with  the 

workstation? 

-  Will  there  be  hard-lines,  or  wireless 
(any  limitations)? 

-  Is  it  digital,  analog,  or  some  combina¬ 
tion? 
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Understanding  the  Requirements 

•  What  are  the  characteristics  of  the  machine 

needed? 

-  Actuators  and  Effectors  (mobility  of 
device,  movement/control  of  objects, 
DOFs,  etc.)? 

-  Sensing  and  Perception  (machine 
interface  to  environment  -  cameras, 
force/torque  transducers,  etc.)? 

-  Data  Management  and  Processing  (pro¬ 
cessor  architecture,  data  buses,  etc.)? 

-  Control  and  Command  Execution  (type 
of  control— position,  rate,  force,  Carte¬ 
sian,  joint,  etc.,  partitioning  of  local/ 
remote  control,  command  authority)? 

-  Operator  Interfaces  (hand  controllers, 
displays,  graphics,  software,  etc.)? 

-  Task  Planning  and  Reasoning  (task 
sequence  development,  situation  analy¬ 
sis  and  reasoning,  replanning,  etc.)? 

•  What  are  impediments  to  using  robotics? 

-  Is  it  lack  of  knowledge  or  under¬ 
standing? 

-  Is  it  the  implementation  costs? 

-  Do  you  need  training? 

Is  new  technology  required? 

•  What  are  the  enablers  for  using  robotics? 

-  Does  it  require  a  proof-of-concept  else¬ 
where? 

-  Does  it  need  to  have  a  minimal  up-front 
cost  to  implement? 

-  Does  the  system  need  to  be  robust? 

-  Is  new  technology  required  (operator 
interface,  Al/reasoning,  improved  ef¬ 
fectors,  etc.)? 

4.0  SUMMARY  AND  CONCLUSIONS 

Robots  fall  short  in  areas  where  humans  excel- 
those  that  require  a  broad  experienced  data  base 
and  the  ability  to  link  disparate,  unexpected 
observations  in  the  field.  Tenets  professing 
this  philosophy  are  listed  in  Exhibit  4-1.  These 
tenets  have  shaped  the  robotics  industry  to  date 
in  terms  of  the  niche  robots  occupy  and  the 
associated  implied  capabilities  and  limitations. 

Three  of  the  most  important  areas  of  research  to 
increase  robotics  applications  by  providing 
humans  with  greater  capability  (listed  in  Exhibit 
4-2)  are  :  1)  more  autonomy  in  robotic  sys¬ 
tems,  2)  greater  mobility  and  capacity  for 
dexterous  manipulation,  and  3)  advances  in  the 
human-machine  interface.  Humans  can  then 


guide  at  any  level  of  control  from  tele-operation 
to  supervised  autonomy,  and  from  both  short 
and  long  distances.  More  effective  robotics 
will  leave  humans  free  to  reason  and  control  at 
the  most  effective  level  for  discovery. 
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Exhibit  1-1:  Topics 


•  Introduction 

•  System  Nomenclature 

•  System  Requirements  and  Design  Criteria 

•  System  Design  Elements 

•  System  Control  and  the  Operator  Interface 

•  Applications 

•  Summary  and  Conclusion 


Exhibit  1-2:  Robotic  System  Trends 


Complex  i 

Unstructured 

^^=ss=  m  ■  . .  ■  ■■ 

v  Non-contact 

Inspection 

Autonomous 

Reasoning 

Machine 

Local/Remote 

Architecture 

Work 

Environment 

Kinematic 
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Controlled 

Telepresence 

Supervised 

Autonomy 

Force- 

Feedback 

Numerically 

Controlled 

Structured 

Master/Slave 

Pick  &  Place 

Teleoperated 

Supervised 

Operator  Interaction 

Robotic  Systems  Are  Evolving  from  Crude  Master/Slave  Devices  | 

to  Sophisticated  Reasoning  Machines  with  Remote  Operator  Telepresence  \ 
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Exhibit  2-1:  Manipulator  System  Functional  Decomposition 


Element 

Function 

Actuators  and  Effectors 

Ability  to  Control  and  Manipulate  Objects  in  the 
Environment 

Sensing  and  Perception 

Machine’s  Interface  to  the  Environment;  Provides 
Information  to  Software  and  Operator 

Data  Management  and  Processing 

Computer  Architecture;  Processors  and  Data 

Buses  That  House  Real-time  Software 

Control  and  Command  Execution 

Software  Algorithms  That  Control  the  System  and 
Execute  Task  Sequences;  Responds  to  Operator 
Inputs  or  Sensing  and  Perception  Data 

Operator  Interface 

Workstation  That  Enables  Operator  to  Interact 
with  Manipulator;  Contains  Hand  Controllers 
Displays  and  Computers 

Task  Planning  and  Reasoning 

Software  That  Enables  Operator  to  Script  Out  and 
Analyze  Situations;  Identifies  Steps  Required  to 
Operate  Safely  and  Accomplish  Mission 

Major  Functions  Required  for  Remotely  Operated  Manipulation  Systems 
Are  Traceable  to  Their  Human  Counterpart _ 


Exhibit  2-2:  Manipulator  System  Components 


Example  for  a  Dexterous  Space  Manipulator 


Telerobot  Body 

•  Head  Cameras 

•  Caging  Mechanism 


Manipulator 

•  Actuator 

•  Inductive  Encoder 

•  Flat  Conductor  Cabling 

•  End-of-Arm  Tooling 

•  Force/T  orque  T  ransducer 

•  Wrist  Camera 

•  Link  Controllers 

Workstation 

•  Hand  Controller 

•  Crew  Restraint 

•  C&DP 


Remotely  Operated  Manipulation  Systems  Are  Composed  of  the  Robot 
Itself  and  the  Operator’s  Workstation  for  Controlling  the  Robot 
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Exhibit  2-3;  Robotic  System  Needs 


External 

•  Operations 
—Remote 
—Local 

•  Power 
—Batteries 
—Host  Vehicle 

•  Mobility  to  and  from  Worksite 
—External  to  Robotic  System 
—Integrated  into  Robotic  System 

•  Data  Exchange  (Data  &  Video) 
—Hardline 

— RF  System 

•  Stability  at  Worksite 

•  Operator  Workstation 
—Hand  Controller(s) 

—Monitors  and  I/O  Interface 

•  User  Interface 
—Human 
—Robotic 


Internal 

•  Data  Processing 
— Internal  to  Robot 
— External  to  Robot 

•  End  Effectors 
—Gripper  and  Tools 
—Articulated  Hand  and  Tools 

•  Manipulator(s) 

—Number  and  DOF 
—Actuators  and  Cabling 

•  Thermal  Control 
— Passive 
—Active 

•  Sensors 
—Video 

—Force  -  Torque 

•  Controls 

— Force  Reflection  or  Feedback 
— Impedance/Compliance 
—Position  (Cartesian)  or  Rate 


All  Robotic  Systems  Have  Basic  Support  Needs  Dependent 
on  Their  Configuration _ 


Exhibit  2-4:  Task-driven  Methodology 


Task  Descriptions 


Task  Analysis 


•  Motion  Requirements 
—  Straight  Line 

—  Curved 
—  Rotations 
—  Distance 
—  Precision 

•  Task  Element  Features 
—  Mass 

—  Volume 

—  Grasping  Interfaces 
—  Tool  Interfaces 

•  Worksite  Characteristics 
—  Clearances 

—  Stiffness 


Kinematic  Simulations 

—  Manipulator  DOFs 

—  Number  of  Arms 

—  Body  Positioning 

Vision  System 

—  Number  of  Cameras 

—  Number  of  Monitors 

—  Lighting  Requirements 

Forces  and  Torques 

—  Sensing  Requirements 

—  Actuator  Sizes 

Collision  Avoidance 

—  Safe  Zones 

—  Clearances 


Robotic  System  Definition 


Manipulator(s)  Features 

—  Size  and  DOFs 

—  Strength  and  Dexterity 

—  Control  Methodology 

Mobility  Implementation 

—  "Body"  Configuration 

—  Support  Systems 

HW/SW  Architecture 

—  Processing  Reqmts 

Human/Machine  Interface 

—  Location 

—  Sensory  Feedback 

—  Complexity 

—  Level  of  Autonomy 


1 


Use  a  Task-driven  Methodology  to  Specify  the  Robotic  System 
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Exhibit  2-5:  Key  Performance  Requirements 


Example  for  a  Dexterous  Space  Manipulator 


Requirement 

Value 

Accuracy 

1  in.,  3  deg  (Autonomous  Tasks  Value  1/4-1/2  of  Spec.) 

Repeatability 

0.1  in.,  0.25  deg  (or  Human  Equivalency) 

0.005  in.,  0.05  deg  (for  NC  Machines) 

Incremental  Motion 

1/5th  the  Value  of  Repeatability 

Tip  Force/Torque  at 
Toolplate 

20  lb  Minimum  Force  in  Any  Direction 

20  ft-lb  Minimum  Torque 

Translation  Rate  at 
Toolplate,  Fully 
Extended 

24  in./sec  Minimum  (No  Load) 

6  in./sec  Minimum  (90  lb  Load) 

Dexterity 

7  DOF  for  Multiple  Poses/Task 
±  90  deg  Wrist  Pitch/Yaw,  ±  360  deg  Wrist  Roll 

Dexterous  Work  Space  4-6  ft  from  Base  of  Manipulator 

r  111  1  ~  ,aassafiW  '  " 

Derive  a  Basis  for  Performance  Requirements  Using  Expected  Tasks 


Exhibit  2-6:  Requirements  Considerations 


System  Performance  Requirements 

•  Derive  a  Basis  for  Accuracy,  Repeatability  and  Incremental  Motion  Based  on 
Expected  Tasks 

•  Define  in  a  Quantitative  Manner  the  Response  of  the  System  from  an  Operator’s 
Standpoint 

•  Base  Requirements  Flow-down  and  Allocation  on  System  Performance  Simulations 


Manipulator  Force/Torque  Requirements 

•  Establish  Realistic  Force/Torque  Requirements  Early  Based  on  Expected  Tasks 

•  Understand  How  Manipulator  Kinematics  Affect  Force,  Torque  and  Power 

•  Define  Point  of  Force/Torque  Application  and  Where  Forces  and  Torques  are 
Measured 

•  Understand  Worksite  Range  and  Locations  at  Which  Force/Torque  Requirements 
Are  Set 

•  Establish  Realistic  Margins,  Not  Just  Minimum  Requirements  and  Understand 
Maximum  Capability  Impacts  on  Safety 


The  Manipulator’s  Required  Performance  Is  the  Key  to 
Ensuring  Mission  Success 
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Exhibit  2-7:  Safety  Requirements 

Example  for  a  Dexterous  Space  Manipulator 


Safety  Requirement 

Ramification 

Two-fault  Tolerance  for  Safing 
System 

•  Addition  of  Backup  Modes/Separate  Paths  for 
Operations 

Two-fault  Tolerance  to 

Inadvertent  Release  of  Hardware 

•  Increased  Size  and  Complexity  of  End  Effector 

•  Additional  Complexity  in  Commanding 

•  Dedicated  Interface  Design  for  Releasable 
Hardware 

Two-fault  Tolerance  to 

Unplanned  Contact  with  the 
Environment 

•  Addition  of  a  Boundary  Management  System 

•  Multiple  Checks  on  Manipulator  Joints  in 
Hardware  and  Software 

Two-fault  Tolerance  to  Applying 
Excessive  Force/Torque  to 
Worksite  during  Planned 

Contact 

•  Addition  of  Emergency  Shutdown  System 

•  Multiple  Paths  in  Hardware  and  Software 

•  Additional  Processors,  Wiring  and  Sensors 

The  Single  Largest  Factor  Driving  System  Design  is  Safety 


Exhibit  2-8:  Worksite  Interface  Considerations 


Worksite  Interface  Considerations 

•  Perform  Kinematic  Analyses  of  All  Tasks  to  Maximize  Manipulator  Dexterity 
(Minimize  Required  Approach  Lengths,  Required  Joint  Travel,  Required  Force  and 
Torque  at  Tool  Interface) 

•  Consider  Structural  Properties  (Stiffness,  Frequency)  of  Worksite 

•  Overall  Viewing  Requirements  Will  Drive  Design  (Camera  Locations  and  Motion, 
Lighting,  Field  of  View) 

•  Consider  Placement  of  Work  Pieces  to  Maximize  Manipulator  Dexterity 

•  Define  the  Work  Piece/Tool  Interface  Early  (Consider  Safety  and  Loads 
Implications) 

•  Implement  Human  Compatibility  Considerations  —  These  Are  Critical  (Safety, 
Interfaces,  Access  and  Clearance,  etc.) 

•  Establish  Viewing/Guiding  Aids  on  Work  Pieces  and  in  Worksite  to  Aid  Operator 
Control 


1 


Worksite  Interfaces  Will  Determine  the  Ease  of  Operations  and  Training 
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Exhibit  2-9:  Considerations  for  Robotic  Compatibility 


Work  Piece  Design  Guidelines: 

•  Single  Arm  Operation 

•  Combined  Handle/Fastener  Interface 

•  Alignment  Guides 

•  Visual  Cues 

•  Soft  Dock 

•  Blind  Mate  Connectors 


Design  for  Robotic  Compatibility  Increases  Utility  of  System 
and  Decreases  Task  Times 


Exhibit  2-10:  Human/Machine  Interface  Considerations 


•  Develop  Human-in-the-Loop  Simulations  Early  to  Assist  in  Requirements 
Derivation  and  Assessment  of  Key  Program  Issues  Related  to  Robotic  Control 

•  Keep  Users  Firmly  in  the  Loop  through  Frequent  and  Periodic  Reviews 

•  Consider  Not  Only  Operational  Workstation  in  Human/Machine  Design  Effort,  But 
Also  Trainer,  Simulator,  Support  Equipment,  etc. 

•  Early  Development  of  Operator  Controls  and  Displays  (Prototype  Simulator)  Is  of 
Great  Value  in  Establishing  Operator  Control  Interface  and  in  Estimating 
Procedural  Timelines 

•  Workstation  Mockups  Are  of  Necessity,  Even  in  the  Age  of  CAD  Designs 


Keep  the  End  Users  Involved  to  Ensure  the  System  Will  Be  Used 


Exhibit  2-11 :  Data  Needs  Checklist 


•  Establish  and  Baseline  Early  in  Design  Data  Requirements  and  Concepts 
(Operator  Display,  Real-time  Support,  Real-time  Anomaly  Resolution,  Post-mission 
Evaluation) 

•  Include  Types  of  Data,  Bandwidth,  and  Accuracy  by  Mission  Phase  for  Planned 
Operations,  and  for  Failures/Contingency  Operations 

•  Establish  and  Understand  Role  of  Operators  and  Logistics  Support  in  Nominal 
Operations,  Trouble-shooting,  and  Contingency  Operations 

•  Establish  Real-time  Data  Needed  to  Be  Able  to  Identify  Failures  to  Correct  Level 

•  Establish  Data  Required  for  Post-mission  Evaluation,  Considering  Both  Normal 
and  Failure/Contingency  Operations 


Data  Requirements  Can  Drive  the  Software  and  Computer  Architecture 


Exhibit  2-12:  Robotic  Manipulator 

Example  of  a  Dexterous  Space  Manipulator 


Human  Equivalency  Drove  the  Design  of  This  Manipulator 
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Exhibit  2-13:  Manipulator  Key  Characteristics 

•  Two-fault  Tolerant  Safety  Design 
—  Runaway  Manipulator 

—  Application  of  Excessive  Forces 
—  Ability  to  Stow 
—  Inadvertent  Release 
—  “Smart”  Computer  Failure 

•  Fully  Embedded  Electronics 

•  Performance  and  Physical  Requirements  Compatible  with  Human  Task 
Characteristics 

•  Human  Compatibility  for  Contingency  Operations 

•  Redundant  Force/T orque  Sensor  for  End  Point  Force  Control 

•  Fault-tolerant  Gripper  with  Force  and  Position  Control 

•  Control  System  Derived  for  Flight  Application 
—  Position  and  Rate  Control 

—  Contact  Stability 
—  Fully  Programmable  Torque  Loops 
—  Impedance  Control 
—  Inertia  Decoupling 


Exhibit  2-14:  Sensing  Alternatives 


•  Seeing 

—  Conventional  Camera  Vision  System  with  Pan,  Tilt  Mechanism 
—  Fish-eye  Vision  System  with  Electro-optical  Pan,  Tilt,  and  Zoom 
—  Capaciflector  Sensor  for  Ranging  and  Imaging 
—  Laser  Range  Finders 

•  Touching 

—  Force/Torque  Sensor  at  the  Tool  Plate 
—  Tactile  Sensors  Imbedded  in  Gripper  Fingers 


Vision  and  Tactile  Sensing  Are  the  Major  Senses  Involved  in  Manipulation 
—  Both  Teleoperation  and  Autonomous  Operation 
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Exhibit  2-15:  Manipulator  Control  Definitions 


•  Kinematics 

—Forward  Kinematics  :  Geometrical  Problem  of  Computing  the  End-Effector 
Position  and  Orientation  Given  a  Set  of  Joint  Angles 

—Inverse  Kinematics  :  Given  the  Position  and  Orientation  of  the  End-Effector, 
Calculate  All  Joint  Angles  Which  Could  Result  in  That  Position  and 
Orientation 


•  Frames/Spaces 

— Joint  Space  :  Representation  of  Manipulator  Position  Relative  to  Joints 

—Cartesian  Space  :  Task  or  Operational  Space  Independent  of  Manipulator 
(i.e.,  Position  of  a  Point  Specified  by  X,  Y,  Z) 

—Tool  Frame  :  Frame  Attached  to  the  Tool  Plate  of  the  Manipulator 


Robotics  Control  Is  Constantly  Concerned  with  the  Relative  Position 
of  the  Manipulator  and  Surrounding  Objects 


Exhibit  2-16:  Major  System-level  Control  Trades 


Trades 

Factors  to  Consider 

Operator  Interface  — 

Teleoperation  to  Supervision 

•  Remoteness  of  Operation 

•  Structure  of  Work  Environment 

•  Resources  Available  (Processing,  Sensing,  etc.) 

Partitioning  of  Functions  — 
Local/Remote 

•  Remoteness  of  Operation  (Time  Delays,  etc.) 

•  Safety  Ramifications 

•  Processing  Capability  at  Both  Locations 

Method  of  Force  Control  — 
Feedback  or  Accommodation 

•  Remoteness  of  Operation 

•  Processing  Capability  at  Both  Locations 

•  Operator-in-the-Loop  Involvement 

Teleoperator  Control  Mode  — 
Master/Slave  or  Cartesian 
Mapping 

•  Processing  Capability  at  Both  Locations 

•  Space  Available  at  Workstation 

Method  of  Incorporating  Vision  — 
Operator  Feedback  or 
Automated  Image  Recognition 

•  Remoteness  of  Operation 

•  Processing  Capability  at  Both  Locations 

•  Work  Space  Structure  (Lights,  Visual  Cues,  etc.) 

Several  Major  Trades  Shape  the  Control  System  Design 
and  They  Are  All  Interrelated 


Exhibit  2-17;  Local/Remote  Partitioning 

An  Example  Based  on  the  JPL  STELER  Laboratory 

Local  -  Operator  Site 

•  User  Macro  Interface  to 
Develop  Task  Scenarios 

•  Setting  of  the  Control  Mode 

(Teleop,  Shared  Control,  _ 

Supervised  Autonomy)  *+ - 37 


Remote  Worksite  Model 
Update  (from  Remote  Sensor 
Data) 

Status  Monitoring  of  the 
System 

Operator  Coached  Machine 
Vision 

Collision  Simulator/Predictor 


Two-way 

Communication 


Remote  -  Robot  Site 

System  Health/Status  Checks 

Position  Trajectory  Generation 

Sensor  Data  (Real  or  Virtual) 
Generation 

Force/Moment  Accommodation 
and  Compliance 

Fusion  of  Command  As 
Specified  by  the  Local  Site 


Proper  Partitioning  of  Functions  Makes  the  System  Less  Sensitive 
to  Communication-related  Destabilizing  Effects 


Exhibit  2-18:  Control  Architecture 

Example  for  a  Dexterous  Space  Manipulator 


NASREM  Primitive  Level 
NASREM  Servo  Level 

_ -J0 

Hardware  Interface^pn 


Force  Feedback  Loop 


Generate 

i 

< 

-  Manipulator  - 

Commands 

1 

*4 

Force 

Reflection 


Autonomous  Position  Command 


Hfematics  /  - -  .apt 

oint  Control 

-  ^Brents 

TorqueLoop 


Position  Control  Loop  Dan 


The  Control  Architecture  Capitalizes  on  the  Capabilities  of  the 
Manipulator  While  Accommodating  Limitations  of  the  Workstation 
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Exhibit  2-19:  Control  Partitioning 

Example  for  a  Dexterous  Space  Manipulator 


A  Functional  Partitioning  of  the  Controls  Is  Based  on  the 
Tasks  and  Required  Performance  of  the  System 


Exhibit  2-20:  Functional  Control  Requirements 

Example  of  a  Dexterous  Space  Manipulator 


Joint  Space  Control 

Provide  Single  and  Multiple  Joint  Control 

Cartesian  Control 

Provide  End-point  Control 

Gripper  Control 

Grasping  Capability 

Teleoperation 

Ground  and  Local 

Autonomous 

Provide  Joint  Space  and  Cartesian  Trajectory 
Capability 

Gain  &  Degree-of-Freedom  Selection 

Independent  Gains  for  Each  Degree  of  Freedom 

Control  and  Object  Frames 

Multiple  Options  of  World,  End  Effector,  Tool 
Plate  and  Camera  Frames  (Absolute  and 

Relative) 

Impedance  Control 

Provide  Position-based  Impedance  Control  to 
Modify  Stiffness  and  Damping  Characteristics 
for  Desired  Frame 

The  Definition  of  Functional  Requirements  Provides  the 
Framework  for  Developing  a  Controls  Architecture 
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Exhibit  2-21 :  NASREM  Hierarchy 


NASA/NIST  Standard  Reference  Model  for  Robotic  Systems 


Level 

Name 

Function 

1 

Servo 

Command  Joint  Motion 

2 

Primitive 

Compute  Dynamically  Efficient  Trajectories 

3 

E-move 

Define  Motion  Pathways  and  Intermediate  Poses 

4 

Object/Task 

Decompose  Actions  Performed  into  Sequences  of 
Motion 

5 

Service  Task 

Partition  Assignments  to  Specific  Systems 

6 

Service  Mission 

Assign  Jobs  and  Service  Resources,  and  Generate  a 
Schedule  of  Activities 

NASREM  Provides  a  Standard  for  Specifying  the 
Hierarchical  Control  Architecture 


Exhibit  3-1 :  Potential  Applications 


Ground 

Space 

General  Manufacturing 

•  Robotic  H/W-S/W  Products 

Batch  Manufacturing 

—  System 

Mining 

—  Manipulators 

Undersea 

—  Subsystems 

Construction 

•  CCDS  Applications 

Health  Services 

•  Servicing 

Agriculture 

—  Payloads 

Materials  Processing 

—  Satellite 

Robotics  Industry 

—  Platforms 

Automotive 

•  Materials  Processing 

Aerospace 

•  Logistics  Support 

Nuclear 

•  Facility  Support 

Service  Industry 

—  Laboratories 

Warehousing 

—  Factories 

Office/Home 

—  Research  Platforms 

Hazardous  Materials  Handling 

•  Construction/Assembly 

Security  Systems 

•  Science 

Energy 

—  Sample  Acquisition 

—  Surface  Surveys 

Benefits  of  Remotely  Operated  Manipulators  Can  Be  Derived  from 
Eliminating  Humans  from  Hazardous  Environments,  Increased  Productivity 
through  Automation  of  Processes,  and  Improving  Existing 
Processes  and  Techniques 
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Exhibit  3-2:  Applications  Questionnaire 


Tonic 

Question 

Understanding  the  Environment 

•  What  Type  of  Operations,  or  Activities, 
Characterize  Your  Industry,  or  Field? 

•  What  Characterizes  the  Environment? 

•  What  Are  the  Constraints  and  Limitations  Placed 
on  Personnel  and  Machines  in  the  Work  Space? 

Understanding  the  Task 

•  What  Are  the  Characteristics  of  the  Work  Space? 

•  What  Are  the  Attributes  of  the  Tasks  Being 
Performed? 

•  Does  It  Require  Manipulating  Objects? 

•  Are  There  Safety  Considerations  Involved? 

Understanding  the  Operator  l/F 

•  If  Robotics  Are  Currently  in  Use,  What  Is  the 
Operator  Involvement? 

•  What  Are  the  Operator  Desires? 

•  How  Will  Communications  Occur  with  the 
Workstation? 

Understanding  the  Requirements 

•  What  Are  the  Characteristics  of  the  Machine 
Needed? 

•  What  Are  Impediments  to  Using  Robotics? 

•  What  Are  the  Enablers  for  Using  Robotics? 

Judicious  Selection  of  the  Appropriate  Robotic  System 
Will  Increase  Benefits  and  Decrease  Training  Required 


Exhibit  4-1:  Human/Machine  Tenets 


Machines  Are  More  Effective  Than  Humans  When  the  Task  Is  Routine 
and  Can  Be  Completely  Specified 

Humans  Process  Complex  Data  Sets  Better,  Work  in  Gestalts 
(Right-brain  Thinking)  and  Deal  with  the  Unpredictable  Better  Than 
Machines 


A  Synergistic  Relationship  Exists  between  Humans  and  Machines 
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Exhibit  4-2:  Robotic  Research  Needs 

•  More  Autonomy  in  Robotic  Systems 

•  Greater  Mobility  and  Capacity  for  Dexterous  Manipulation 

•  Advances  in  the  Human/Machine  Interface 

High  Payoff  Exists  for  Increased  Robotic  Applications  by 
Giving  Humans  More  Capability 
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CONTROL  AND  OPERATION  OF  SPACE  MANIPULATOR  SYSTEMS 


C.  P.  Trudel1,  D.  G.  Hunter2,  M.  E.  Stieber3 
Canadian  Space  Agency/Agence  spatiale  canadienne 
6767  route  de  l'Aeroport,  Saint-Hubert  Quebec,  Canada. 


SUMMARY 

As  manned  space  flight  passes  into  its  fourth 
decade,  many  of  the  activities  in  space  appear  to 
be  almost  routine.  However,  the  space 
environment  is  exceedingly  hostile  to  humans  and 
necessitates  substantial  effort  and  funding  to 
provide  an  infrastructure  to  support  human  life. 
Increased  attention  is  therefore  being  directed 
toward  the  application  of  robotics  technology  to 
more  effectively  carry  out  tasks  in  space  which,  up 
to  now,  have  been  carried  out  by  human  Extra- 
Vehicular  Activity  (EVA),  or  which  could  not  be 
carried  out  because  of  human  limitations  or 
limitations  of  available  equipment.  Manipulator 
systems  are  better  suited  to  operation  in  the 
external  space  environment  and  can  augment  or 
replace  the  human  presence.  They  can  also  play 
an  important  role  in  the  spacecraft  Intra- Vehicular 
Activity  (IV  A)  to  replace  some  of  the  human  tasks. 
To  date  few  robots  have  been  developed  for  space 
applications;  however,  as  human  activity  increases 
in  earth  orbit  and  beyond,  robotic  systems  will 
play  an  increasingly  vital  role.  Control  and 
operation  of  space  manipulator  systems  are 
addressed  with  emphasis  on  those  designed  for  the 
external  space  environment.  Applications  of  the 
technology  are  discussed  in  the  context  of  the 
Mobile  Servicing  System  (MSS)  being  developed 
for  the  international  Space  Station.  The  Mobile 
Servicing  System  incorporates  two  manipulator 
systems,  the  Space  Station  Remote  Manipulator 
System  (SSRMS)  and  the  Special  Purpose 
Dexterous  Manipulator  (SPDM). 


1  Control  Systems  Specialist,  Space  Station  Program 

2  Manager,  Automation  and  Robotics  Engineering,  Space 
Station  Program. 

3  Manager,  Control  Systems  and  Simulation,  Space  Station 
Program. 


The  modes  of  operation  and  selectable  control 
features  are  discussed  with  some  of  the  more 
advanced  features  demonstrated  by  simulation  and 
laboratory  test  results. 


1.0  INTRODUCTION 

The  exploration  and  practical  utilization  of  space 
presents  exceptional  engineering  challenges.  Many 
of  these  challenges  are  rooted  in  the  extremely 
hostile  space  environment.  In  order  for  humans  to 
venture  into  space  it  is  necessary  to  encapsulate 
them  in  a  space  vehicle  which  provides  an 
environment  similar  to  that  on  earth  and  shields 
them  from  the  hazards  of  the  space  environment. 
Any  Extra- Vehicular  Activity  (EVA)  for 
construction,  maintenance  or  repair  outside  the 
space  vehicle  requires  the  use  of  a  space  suit  to 
extend  that  safe  environment  to  the  worksite. 
EVA  operations  require  extensive  preparation  and 
training  and  involve  many  hazards  with  attendant 
high  mitigating  cost  for  simulators,  under-water 
facilities,  etc. 

Robotic  devices  can  play  an  important  role  in 
improving  productivity  and  reducing  the  hazards  of 
humans  in  space. 

The  International  Space  Station  will  provide  a 
permanent  base  in  low  earth  orbit  for  conducting 
space  research  in  such  areas  as  astronomy, 
materials,  and  life  sciences.  The  Space  Station  is 
shown  in  Figure  1 . 

The  Mobile  Servicing  System  (MSS)  being 
developed  by  Canada  for  the  International  Space 
Station  is  the  first  robotic  system  that  has  been 
assigned  very  diverse  tasks  in  space,  and  represents 
the  current  state-of-the-art  in  space  robotics 
hardware. 


Presented  at  an  AGARD  Lecture  Series  on  ‘Advanced  Guidance  and  Control  Aspects  in  Robotics’ ,  June  1994. 
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Figure  1.  International  Space  Station 

The  Space  Station  Remote  Manipulator  System 
(SSRMS)m,  and  a  dual-ami  robot,  the  Special 
Purpose  Dexterous  Manipulator  (SPDM)121  provide 
the  robotic  capabilities  of  the  Mobile  Servicing 
System.  The  two  manipulator  systems  can  work 
independendy  of  each  other,  or  they  can  work 
together  with  the  SPDM  attached  to  the  end  of  the 
SSRMS.  The  SSRMS  is  designed  to  handle  large 
payloads  and  the  SPDM  is  designed  to  handle 
small  payloads  and  perfonn  dexterous  tasks. 

The  control  and  operation  of  the  total  MSS  system 
will  be  described  briefly  followed  by  separate  and 
more  detailed  discussions  of  the  SSRMS  and 
SPDM.  Emphasis  is  placed  on  the  modes  of 
operation  and  control  and  some  of  the  built-in 
features  designed  to  enhance  the  capability  of  the 
manipulators  and  reduce  the  workload  on  the 
human  operators.  Included  is  a  brief  discussion  of 
a  concept  presendy  being  developed  whereby  an 
option  to  operate  the  MSS  from  a  ground-based 
control  station  is  incorporated.  This  feature  would 
further  increase  productivity  by  allowing 


manipulator  operation  by  a  ground-based  or  an  in- 
orbit  operator. 

2.0  THE  ROLE  OF  ROBOTICS  IN  SPACE 

Manipulator  systems  can  be  designed  to  perform 
many  of  the  tasks  which  are  perfonned  by  humans 
in  the  external  space  environment.  They  can 
replace  or  at  least  augment  the  human  activity  in 
many  areas,  and  can  perform  some  tasks  which 
humans  cannot  perform  in  space  due  to  physical 
limitations.  Manipulators  can  be  designed  for 
extended  exposure  to  the  space  environment,  with 
small  operating  energy  consumption  and  very  small 
"keep-alive"  energy  consumption  when  they  are 
idle.  The  energy  that  is  consumed  is  usually 
electrical  and  can  be  generated  using  on-orbit 
systems. 

Manipulators  can  be  designed  for  optimum 
performance  of  specific  tasks,  or  a  range  of  tasks. 
Large  and  powerful  manipulators  can  be  used  to 
manoeuvre  large  payloads  or  assist  in  the  assembly 
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of  large  structures.  Smaller  manipulators  can 
provide  more  dexterous  capabilities  to  replace 
electronic  packages,  mate/demate  connectors, 
open/close  hatches,  etc.,  significantly  reducing  the 
requirements  on  the  very  expensive  and  hazardous 
human  EVA  activity.  Just  as  earth-bound 
machines  are  designed  to  augment,  extend  or 
replace  human  capabilities,  increase  safety  and 
efficiency,  reduce  stress,  etc.,  intelligent  space 
manipulators  and  robots  can  provide  these  features 
in  space  operations. 

Few  robotic  devices  have  been  developed  for  space 
applications  to  date.  The  Shuttle  Remote 
Manipulator  System  (SRMS)  or  "CAN  AD  ARM” 131 
developed  by  Canada  for  the  U.S.  space  shuttle 
program  has  been  in  operation  for  approximately 
10  years.  Other  robotic  devices  have  been  used  to 
obtain  soil  samples  on  Mars  and  on  the  moon.  In 
1993,  the  Robotics  Technology  Experiment, 
(ROTEX)[4i  was  flown  during  the  Spacelab  D-2 
mission  and  successfully  demonstrated  advanced 
space  automation  and  robotics  technologies.  The 
ROTEX  flight  system  comprised  a  small  6  degree- 
of-freedom  manipulator  inside  the  Spacelab  with  a 
robot  workcell  with  various  fixtures  for 
experimentation,  and  a  control  station  with  3-D 
video  display  and  6  degree-of-freedom  hand 
controller.  The  manipulator  could  be  controlled 
from  the  Spacelab  or  from  a  control  station  on  the 
ground. 

Several  space  manipulator  systems  are  in 
development:  In  addition  to  the  MSS  being 
developed  by  Canada  for  the  international  Space 
Station,  the  External  Robotic  Arm  (ERA)  is  being 
developed  by  Europe,  and  two  manipulator  systems 
are  being  developed  by  Japan  for  use  with  the 
Japanese  Experiment  Module  (JEM)  on  the 
international  Space  Station;  the  JEM  Remote 
Manipulator  System  (JEMRMS)  and  the  Small 
Fine  Arm  (SFA). 

As  the  intelligence  of  space  robotic  systems 
increases,  and  as  human  confidence  in  their 
capabilities  and  safety  builds,  the  scope  of  their 
use  will  no  doubt  increase.  The  autonomy  of  their 
operation  will  also  naturally  increase.  Whereas 


present  day  robotic  devices  in  space  are  typically 
commanded  at  a  fairly  primitive  level,  i.e.,  "move 
in  direction  x",  or  "perform  automatic  sequence  of 
motions  a-b-c".  in  the  future,  more  intelligent 
devices  with  sophisticated  sensing  systems  might 
respond  to  "replace  electronic  package  'a'  on  device 
'xyz'".  The  planning  of  the  detailed  operations  and 
the  problems  of  dealing  with  non-routine 
occurrences  during  operation  will  tend  to  shift 
away  from  the  human  planners  and  operators  to  the 
robotic  systems  as  the  intelligence  of  the  robotic 
devices  increases. 

The  design  and  operating  features  of  the  Mobile 
Servicing  System  will  now  be  discussed. 

3.0  MOBILE  SERVICING  SYSTEM 

The  Mobile  Servicing  System  is  Canada's 
contribution  to  the  international  Space  Station  and 
will  play  a  predominant  role  in  the  assembly  and 
maintenance  of  the  Space  Station.  The  MSS  is 
being  developed  by  Spar  Aerospace  Limited  for 
the  Canadian  Space  Agency.  It  is  designed  to  be 
repaired  and  maintained  in  orbit  and  will  have  the 
capability  to  handle  payloads  with  mass  properties 
up  to  and  including  those  of  a  fully  loaded  shuttle 
orbiter.  The  MSS  will  also  incorporate  the 
capability  to  relocate  its  manipulator  systems  on 
the  space  station. 

During  assembly  and  operation  of  the  Space 
Station,  many  operations  in  the  external 
environment  will  be  required.  These  operations 
will  be  carried  out  by  means  of  Extra-Vehicular 
Activity  of  the  Station  crew  and  by  manipulator 
systems. 

The  MSS  is  being  developed  to  satisfy  the 
following  functions  for  the  Space  Station: 

-  Space  Station  Assembly 

-  Space  Station  external  maintenance 

-  transportation  on  the  Space  Station 

-  servicing  of  external  payloads 

-  deployment  and  retrieval  functions 

-  EVA  support 
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The  MSS  comprises  a  space  segment  and  a  ground 
segment.  In  order  to  cany  out  this  diverse  set  of 
functions,  the  space  segment  design  consists  of 
four  elements: 

-  the  Mobile  Servicing  Centre  (MSC), 

-  a  Special  Purpose  Dexterous  Manipulator 
(SPDM), 

-  a  MSC  Maintenance  Depot  (MMD), 

-  a  dedicated  control  station. 

The  ground  segment  is  made  up  of  facilities  to 
provide  engineering  support  to  operation,  training, 
task  verification  and  sustaining  engineering 
functions.  The  MSS  Architecture  is  shown  in 
Figure  2. 

The  MSC,  depicted  in  Figure  3  with  the  SPDM 
attached,  comprises  the  Space  Station  Remote 
Manipulator  System,  an  operations  platfonn  called 
the  MSC  Base  System  (MBS),  and  a  United  States 
supplied  Mobile  Transporter  (MT).  The  MT  is 


designed  to  move  along  rails  on  the  Space  Station 
truss  to  transport  the  MBS,  SSRMS,  SPDM  and 
other  payloads.  The  capture,  manipulation  and 
berthing  of  large  payloads  is  performed  by  the 
MSC  using  the  SSRMS. 

MSS  functions  requiring  dexterous  capabilities  are 
satisfied  by  the  dual-arm  SPDM.  The  SPDM  will 
play  a  role  in  the  Space  Station  maintenance, 
assembly,  and  in  payload  servicing.  It  is  capable 
of  operating  from  fixtures  on  the  MBS,  from  the 
end  of  the  SSRMS,  or  from  appropriate  fixtures  on 
other  structure. 

Because  it  is  permanently  installed  on  the  Space 
Station,  repair  and  maintenance  of  the  MSS  must 
be  perfonned  on  orbit.  The  MBS  is  designed  to 
provide  appropriate  positioning  of  the  SSRMS  and 
SPDM  for  maintenance.  The  MMD  is  configured 
to  store  a  complement  of  critical  MSS  replacement 
components  and  tools. 


SPACE  STATION  PRESSURE  ED  VOLUME 
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•VIDEO  AND  SVNCH 

POWER  CHANNELIZATION 


P,  -  PRIME  POWER 
P,  •  REDUNDANT  POWER 
P,  -  BACKUP  POWER 


SPDM  ON  SSRMS  LEE 


Figure  2.  Mobile  Servicing  System  Architecture 
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Spacs  Station 

Manipulator 

(SSflMS) 


Special  Purpooa 
Oaxtofou*  Manipulator 
(SPDM) 


Figure  3.  Mobile  Servicing  Centre  with  SPDM 


A  dedicated  control  station  is  used  by  the  MSS 
operator  to  command  and  monitor  MSC  and 
SPDM  operations  from  a  pressurized,  shirt-sleeve 
environment  on  the  station.  The  control  station  is 
shown  in  Figure  4.  The  control  station-to-operator 
interface  includes  a  number  of  displays  for  video 
views  of  operations,  a  command  and  control 
display  providing  graphical  and  numerical 
information  and  soft  keys  which  are  activated  by  a 
trackball  input,  two  hand  controllers  for  inputting 
manual  manipulator  commands,  and  a  keyboard. 

Concepts  are  presently  being  developed  for  an 
additional  control  station  to  be  included  in  the 
MSS  Ground  Segment.  This  control  station  would 
have  a  similar  operator  interface  but  would  include 
some  additional  features  to  accommodate 
communications  delays  and  other  problems 
associated  with  remote  operation.  This  control 
station  would  provide  the  optional  capability  to 
operate  the  MSS  from  the  ground. 

4.0  SPACE  STATION  REMOTE 
MANIPULATOR  SYSTEM 

The  SSRMS,  as  shown  in  Figure  5,  is  a  7-joint 


-  Programmable 
Display  Pushbutton 

-  ECLSS  Duct  Closeout 


-  Node  Bulkhead 


Figure  4.  MSS  Control  Station 


symmetrical  manipulator  approximately  17  metres 
in  length  when  fully  extended.  A  symmetrical 
arrangement  of  joints  and  a  Latching  End-Effector 
(LEE)  at  each  end  allows  either  end  to  attach  to 
payloads,  or  to  serve  as  a  base  for  the  SSRMS 
providing  that  an  appropriate  Power  Data  Grapple 
Fixture  (PDGF)  interface  is  available.  The  LEE 
along  with  the  PDGF  interface  are  shown  in  Figure 
6.  The  LEE  incorporates  a  snare  mechanism,  a 
rigidizing  carriage  mechanism,  a  latching  system 
and  umbilical  connection.  The  snare  mechanism  is 
designed  to  snare  the  protruding  probe  of  the 
grapple  fixture.  After  the  snare  is  closed,  the 
carriage  containing  the  snare  mechanism  and  the 
snared  probe  is  drawn  into  the  LEE  until  the 
grapple  fixture  base  plate  is  in  full  contact  with  the 
face  of  the  LEE  with  a  specified  preload.  If  a 
higher  stiffness  interface  is  required,  or  if  the 
power  and  data  is  to  be  transferred  across  the 
interface,  a  latching  mechanism  is  activated,  and  an 
umbilical  connector  is  engaged  with  its  mating 
connector  on  the  Grapple  fixture.  Power,  data  and 
video  may  be  passed  through  the  SSRMS  to 
operate  the  SPDM  while  attached  to  the  SSRMS 
LEE  or  to  support  the  keep-alive  power,  telemetry 
and  command  requirements  of  payloads  attached  to 
the  LEE.  Four  video  cameras  are  mounted  on  the 
SSRMS,  one  fixed  camera  at  each  end  effector, 
and  one,  along  with  a  pan  and  tilt  unit  on  either 
side  of  the  elbow  joint  on  the  main  booms.  A 
light  is  provided  with  each  of  the  cameras. 
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Figure  5.  Space  Station  Remote  Manipulator  system 


Figure  6.  Latching  End  Effector  and  Power  Data  Grapple  Fixture 
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Figure  7.  Shuttle  Berthing  Operation  using  SSRMS 


Although  the  majority  of  SSRMS  operations  will 
be  perfonned  from  the  Mobile  Base  System,  the 
symmetrical  arrangement  of  joints  and  LEEs  gives 
the  SSRMS  a  self-relocating  capability.  PDGFs 
mounted  strategically  along  the  Space  Station 
allow  the  SSRMS  to  step  from  one  PDGF  to 
another  to  access  areas  which  cannot  be  reached 
when  operating  from  the  MBS. 

The  7  joints  of  the  SSRMS  are  arranged  in  clusters 
of  three  joints  near  each  end  of  the  manipulator  to 
act  as  a  "wrist"  and  "shoulder"  respectively,  with 
an  additional  joint  at  the  midpoint  "elbow" 
position.  Starting  from  either  end,  the  joint 
sequence  is  roll,  yaw,  pitch,  pitch,  pitch,  yaw,  roll. 
All  joints  are  identical  and  have  a  range  of  travel 
of  +/-270  degrees.  Because  the  number  of  joints 
exceeds  the  6  degrees-of-freedom  in  which  the 
manipulator  tip  is  being  controlled,  the  manipulator 
is  classified  as  kinematically  redundant.  The 
kinematic  redundancy  increases  the  operational 
flexibility  helping  to  avoid  kinematically  singular 
configurations. 


4.1  Major  SSRMS  Performance  Requirements 

Operations  using  the  SSRMS  involve  the  handling 
and  positioning  of  a  wide  range  of  payload  shapes 
and  mass  properties.  The  mass  range  of  payloads 
which  can  be  handled  by  the  SSRMS  is  from  zero 
(i.e.,  no  payload)  up  to  116,000  Kg,  which  is 
representative  of  a  fully  loaded  shuttle  orbiter. 

A  computer  generated  view  of  the  SSRMS 
performing  a  manoeuvre  to  berth  the  shuttle  orbiter 
to  the  Space  Station  is  shown  in  Figure  7. 

The  ability  to  stop  the  SSRMS  and  its  attached 
payload  within  a  known  distance  when  the 
manipulator  is  commanded  to  stop,  or  when  an 
emergency  stop  is  initiated,  is  of  crucial 
importance  to  avoiding  damage  to  the  SSRMS  or 
other  equipment.  The  stopping  distance/angle 
requirements  of  the  manipulator  tip  and  payload  set 
the  maximum  manoeuvring  rate  requirements  for 
various  payload  sizes  and  influence  the  sizing  of 
joint  actuators,  drivetrains,  emergency  braking 
devices,  etc.  By  manoeuvring  within  the  maximum 
rate,  stopping  can  always  be  guaranteed  within  the 
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specified  distance  (usually  0.3  m)  even  in  the  event 
of  failures. 

Large  payloads  must  be  manoeuvred  at  very  low 
rates  to  satisfy  stopping  requirements.  In  order  to 
ensure  good  controllability  at  these  low  rates,  the 
control  system  must  exhibit  high  rate  accuracy 
(low  drift).  SSRMS  drift  rate  is  specified  to  be 
better  than  3  mm/sec. 

Assembly  and  berthing  operations  require  accurate 
positioning  of  payloads  to  accommodate  the 
capture  range  of  mating  devices  and  berthing 
mechanisms.  The  open-loop  accuracy  of  the 
SSRMS  (based  on  joint  angle  measurements)  is 
specified  as  4.5  cm/  0.7  deg.  The  resolution  of 
motion  capability  is  specified  to  be  better  than  1 
cm  and  0.1  deg.  Significant  improvement  on  the 
open-loop  positioning  is  therefore  possible  if  good 
visual  cues  are  available  in  manual  modes,  or  if 
vision  system  feedback  is  used  in  automatic 
modes. 

Constrained  motion  operations  during  assembly 
operations  or  payload  berthing  require  that 
interface  loads  are  limited  to  prevent  jamming  or 
damage  to  equipment.  Static  loads  at  the  SSRMS 
tip  are  specified  not  exceed  1000  N  and  3100  N-m 
during  constrained  motion  operations.  There  is 
also  a  requirement  for  the  SSRMS  to  be  capable  of 
actively  limiting  static  loads  at  the  tip  by  means  of 
force-moment  accommodation,  to  selectable  values 
in  the  range  of  0  to  445  N/407  N-m. 

5.0  SPECIAL  PURPOSE  DEXTEROUS 
MANIPULATOR 

The  SPDM  is  shown  in  Figure  8.  The  robot  is 
made  up  of  two  major  assemblies;  a  base  segment 
and  two  manipulator  anus.  The  base  segment  has 
a  latching  end  effector  at  one  end,  which  is  the 
same  as  the  SSRMS  LEE,  and  a  PDGF  at  the  other 
end  which  is  compatible  with  the  SSRMS  LEE. 
Thus  die  SPDM  can  be  attached  to  the  SSRMS  to 
act  as  an  extension  to  the  SSRMS,  or  it  may  be 
attached  to  a  PDGF  on  the  MBS  or  at  other 
locations  on  the  Space  Station.  A  shoulder 
structure  supporting  the  two  arms  is  attached  to  the 


base  on  the  PDGF  side  of  a  body  joint.  This 
allows  the  upper  body,  including  the  arms  to  rotate 
relative  to  the  LEE.  A  platfomi  is  attached  to  the 
LEE  side  of  the  base  with  accommodation  for  the 
storage  of  tools  and  transport  of  ORUs. 

The  two  SPDM  arms  are  identical  7-joint 
manipulators  with  a  clusters  of  3  joints  at  the 
shoulder  and  near  the  tip,  with  a  pitch  joint  at  the 
elbow  position  near  the  midpoint  of  each  ann.  The 
arms  have  the  same  joint  sequence  as  the  SSRMS 
and  are  therefore  described  by  similar  kinematic 
equations.  The  tip  of  each  ann  is  equipped  with 
an  ORU-Tool  Changeout  Mechanism  (OTCM)  and 
the  wrist  of  each  arm  contains  a  six  axis  force- 
moment  sensor.  The  OTCM  is  depicted  in  Figure 
9.  It  incorporates  a  parallel  jaw  gripper  compatible 
with  standard  H  and  micro  fixtures,  an  extendable 
7/16  inch  socket  drive,  a  camera  with  a  two  stop 
zoom  lens,  two  lights,  and  an  extendable  umbilical 
mechanism.  In  addition  to  the  OTCM  cameras, 
there  are  cameras  with  pan  and  tilt  units  on  the 
fore  part  of  each  arm  and  on  each  side  of  the  body. 
There  is  also  one  fixed  camera  on  the  SPDM  base 
I  EE,  for  a  total  of  7  cameras.  Each  camera  is 
collocated  with  a  light. 

An  interface  is  provided  on  the  SPDM  to  attach  an 
ORU  carrier  with  the  capability  of  transporting  up 
to  six  ORUs,  allowing  several  ORUs  to  be 
replaced  during  a  single  mission. 

5.1  Major  SPDM  Performance  Requirements 

The  SPDM  is  employed  for  numerous  dexterous 
operations  involved  with  assembly  and 
maintenance  such  as  handling  and  replacing  ORUs, 
connecting/disconnecting  utilities,  attaching  covers, 
opening  hinged  doors,  and  performing  operations 
with  special  tools.  Operations  are  performed  while 
the  SPDM  is  mounted  through  its  PDGF  interface 
on  the  end  of  the  SSRMS,  or  through  its  LEE 
interface  on  the  MBS  or  on  the  Space  Station. 
While  operating  from  the  SSRMS,  the  OTCM  of 
one  of  the  SPDM  manipulators  is  usually  attached 
to  a  standard  H  interface  on  structure  near  the 
worksite  to  stabilize  the  SPDM  while  the  other  arm 
performs  the  dextrous  task.  Some  operations 
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Figure  8.  Special  Purpose  Dexterous  Manipulator 


Figure  9.  ORU/Tool  Changeout  Mechanism 

require  the  use  of  both  arms  without  the  added 
stabilization.  Some  of  the  performance 
specifications  must  therefore  take  into  account  the 
significant  variation  in  support  stiffness  for  these 
operating  configurations. 

Figure  10  is  a  computer  generated  graphics  view 
showing  the  SPDM  preparing  to  remove  a  battery 


box  from  a  carrier.  Note  that  the  SPDM  base 
PDGF  is  attached  to  the  SSRMS  LEE  and  that  the 
OTCM  of  one  of  the  SPDM  arms  is  attached  to  a 
grasp  interface  on  a  nearby  beam  for  added 
stability. 

The  range  of  design  case  payloads  which  may  be 
handled  by  the  SPDM  is  from  zero  (no  payload)  to 
600  Kg.  with  maximum  dimensions  of  1  metre 
square. 

As  with  the  SSRMS,  the  stopping  distance 
requirements  and  maximum  interface  loads 
requirements  determine  the  maximum  translational 
and  rotational  manoeuvring  rates  and  influence  the 
design  of  actuators,  drivetrain  and  emergency 
braking  systems.  Maximum  stopping 
distance/angle  during  payload  manoeuvring 
operations  are  specified  as  5  cm/1.75  deg.  During 
ORU  removal  and  insertion,  there  is  also  a 
requirement  on  the  maximum  impact  energy  at  the 
interface,  which  also  determine  the  maximum  rates 
for  this  class  of  operation. 

During  constrained  motion  operations  such  as 
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Figure  10.  SPDM  Performing  Battery  Box  Changeout 


insertion  of  an  ORU  into  guides,  the  positioning 
accuracy  and  the  accuracy  of  motion  is  of  utmost 
importance.  The  open-loop  positioning  accuracy 
of  the  SPDM  manipulators  is  specified  to  be  better 
than  6  mm  and  1  degree.  Specification  of  smaller 
minimum  positioning  increment  and  specifications 
on  the  resolution  of  visual  cues  and  on  the  AVF 
allow  improvement  over  the  open-loop  accuracy  by 
operator  adjustments  in  manual  modes,  or  through 
the  use  of  AVF  supported  automatic  modes.  The 
accuracy  of  motion  or  minimum  tip  velocity  the 
manipulators  is  specified  to  be  2  mm/sec  and  0.06 
deg/sec. 

In  order  to  accomplish  ORU  insertion  or  removal, 
the  SPDM  manipulators  are  required  to  be  capable 
of  applying  insertion  and  removal  forces  in  the 
range  of  22  to  111  N  for  up  to  30  seconds. 

Maximum  interface  loads  between  the  OTCM  and 
payload  are  specified  as  222  N  and  169.5  N-m  to 
avoid  damage  to  equipment.  There  is  also  a 
requirement  on  the  force-moment  accommodation 
feature  to  actively  limit  the  forces  and  moments  to 
selectable  values  within  that  specified  range. 


6.0  VISION  SYSTEM 

A  major  operational  component  of  the  MSS  control 
equipment  is  the  Artificial  Vision  Unit  (AVU) 
which  will  play  an  important  role  in  the  manual 
and  automatic  operation  of  both  the  manipulators 
and  cameras.  In  one  mode  the  AVU  will  provide 
data  to  the  MSS  operator  in  the  form  of  graphical 
and  textual  displays.  In  another  mode  the  AVU 
will  provide  outputs  which  can  be  used  in  real-time 
to  implement  automated  tracking  modes  in  the 
MSS  manipulators.  The  AVU  is  capable  of 
utilizing  the  output  of  any  of  the  MSS  or  Space 
Station  cameras. 

In  October  1992,  on  Shuttle  mission  STS-52,  a 
machine  vision  system,  called  the  Space  Vision 
System  (SVS),  flew  in  space  for  the  first  time151. 
The  flight  tests  verified  the  fundamental  principles 
of  using  real-time  photogrammetry  techniques  on 
a  flexible  space-based  platform  such  as  a  shuttle  or 
space  station.  The  photogrammetry  techniques 
used  are  the  core  of  the  AVU.  A  series  of 
experiments  were  conducted  covering  many  of  the 
baseline  requirements  of  the  AVU,  as  well  as 
many  of  the  features  related  to  manipulator  tasks. 
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It  was  shown,  using  SVS  displays  and  the  SRMS, 
that  an  operator  could  control  the  manipulator  with 
substantially  more  precision  than  is  possible 
without  the  displays. 

7.0  MSS  MANIPULATORS  CONTROL 
SYSTEM  DESIGN  AND  OPERATION 

It  is  evident  from  the  above  descriptions  of  the 
SSRMS  and  the  SPDM  manipulators  that  the 
manipulators  are  of  a  similar  kinematic 
configuration  and  that  their  tasks  and  perfonnance 
requirements  are  similar  in  many  ways,  although 
on  a  different  scale.  This  comes  about  because  of 
the  general  purpose  nature  of  both  systems, 
although  they  are  intended  to  operate  in  different 
payload  ranges  and  with  different  degrees  of 
dexterity.  Indeed,  the  design  of  the  control 
systems  for  the  SSRMS  and  for  the  SPDM 
manipulators  are,  in  principle,  almost  identical.  In 
most  cases,  only  parameter  values  for  the  system 
and  control  laws  differ.  The  description  of  the 
control  system  design  will  therefore  be  discussed 
with  respect  to  both  the  SSRMS  and  SPDM. 
Differences  will  be  pointed  out. 

The  MSS  manipulators  are  controlled  from  a 
control  station  in  the  pressurized  Space  Station 
environment  (Figure  4).  A  requirement  to  include 
the  capability  to  control  the  MSS  from  a  ground- 
based  command  source  through  a  radio  link  has 
recently  been  baselined.  The  concept  which  is 
presently  being  studied  is  discussed  in  Reference 
6  and  is  depicted  in  Figure  1 1.  The  ground  control 
station  human  interface  would  be  similar  to  that 
on-orbit,  but  with  special  features  to  deal  with 
problems  unique  to  controlling  from  a  distant 
source,  namely  communications  delays  of  several 
seconds,  and  reliance  on  a  software-based  world 
model  for  the  manipulator  operations.  The  concept 
includes  the  possibility  of  using  predictive 
simulation  with  animation  and  graphics  to  provide 
the  operator  with  a  non-dclayed  representation  of 
the  manipulator  response.  Concepts  are  also  being 
developed  to  update  the  world  model  in  real-time 
in  order  to  better  predict  the  exact  location  of 
objects  in  the  manipulator  workspace.  This 
information  is  critical  when  performing  contact,  or 


Figure  11. 

MSS  Ground  Control  Functional  System  Concept 


close  proximity  operations. 

A  block  diagram  of  the  SPDM  control  architecture 
for  the  on-orbit  control  concept  is  shown  in  Figure 
12.  Additional  detail  of  the  MSS  architecture  may 
be  found  in  Reference  [7].  The  control 
architecture  for  the  SSRMS  is  similar  to  that  in 
Figure  12  but  would  not  contain  blocks  for  the 
second  arm,  body  joint  or  SPDM  LEE.  References 
to  the  OTCM  would  be  replaced  by  LEE,  and 
OEU  by  LEU  for  the  SSRMS.  This  control 
concept  allows  the  MSS  manipulators  to  be 
operated  in  a  number  of  human-in-the-loop  and 
automatic  modes. 

7.1  Operating  Control  Modes  and  Features 

The  control  modes  and  features  described  here  for 
the  MSS  manipulators  reflect  the  state-of-the-art 
for  the  current  generation  of  external  space 
robotics.  Because  of  the  unique  problems 
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Figure  12.  SPDM  Control  System  Architecture 


encountered  in  the  space  environment,  it  is 
believed  that  many  of  the  characteristics  of  die 
MSS  manipulator  modes  and  features  would  be 
typical  of  the  modes  and  features  of  most  general 
purpose  manipulators  designed  to  perform  a  wide 
variety  of  tasks  in  space.  Simpler,  special  purpose 
manipulators  would  not  incorporate  all  of  these 
features,  or  may  require  other  special  features  to 
suit  their  specific  purpose. 

Control  Modes 

The  control  modes  for  the  SSRMS  and  SPDM 
manipulators  are  described  briefly  in  Table  1 . 
Motion  commands  to  the  manipulators  can  be 
resolved  motion  commands  or  joint-by-joint 
commands.  In  resolved  motion  operation, 
translational  and  rotational  velocity  inputs  from  an 
operator  or  automation  software  provide  commands 
in  a  selected  Point-of-Resolution  (POR)  reference 
frame.  The  origin  of  the  POR  frame  is  typically 
chosen  to  be  the  ami  tip  or  a  point  of  interest  on  a 
payload  or  tool.  Inverse  kinematic  equations  in  die 


TABLE  1.  MSS  MANIPULATOR  CONTROL  MODES 


|  CONTROL  MODE 

COMMANDED  MOTION 

|  HUMAN  IN  THE  LOOP  CONTROL  MODES  j 

Manual  Augmented 

Manipulator  receives  hand  controller  data 
and  moves  selected  POR  at  the  specified 
rate. 

Single  Joint  Rate 

Movement  on  joint-by-joint  basis.  Other 
joints  in  Joint  Position  Hold. 

Back-up  Drive 

Movement  on  joint-by-joint  basis  using 

BDU.  Brakes  on  for  ail  other  joints. 

|  AUTOMATIC  TRAJECTORY  | 

Operator  Commanded  POR 

Control  of  POR  from  its  current  position 
to  operator-specified  destination. 

Operator  Commanded  Joint  Position 

Control  of  joints  from  current  positions  to 
operator-specified  destinations. 

Prestored  POR  Auto  Sequence 

POR  commanded  along  predefined 
trajectory. 

Prestored  Joint  Position  Auto 
Sequence 

Joints  move  in  a  predefined  joint  position 
sequence. 

AVF  Supported  Tracking 

Manipulator  responds  to  relative  position 
information  generated  by  AVF  and 
provided  by  MCCF. 

ann  control  software  resolve  the  POR  commands 
into  individual  joint  commands  to  the  joint  servos. 
For  joint-by-joint  operation,  commands  in  the 
individual  joint  axes  are  input  directly  to  the  joints. 
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Because  of  the  kinematic  redundancy  of  the  7-joint 
manipulators,  the  inverse  kinematics  equations  in 
the  control  laws  have  a  multiplicity  of  solutions. 
The  result  is  uncertainty  in  the  ann  configuration 
for  a  specific  tip  position  and  orientation.  This 
problem  is  handled  in  the  control  laws  by 
providing  an  option  to  apply  an  additional 
constraint  to  control  one  of  the  joints  (usually  a 
shoulder  joint)  to  a  constant  position  unless  it  is 
required  to  move  to  avoid  a  kinematic  singularity. 

Although  resolved  motion  operation  is  the  primary 
means  of  controlling  the  manipulators,  joint-by¬ 
joint  operation  is  an  option  in  both  human-in-the- 
loop  and  automatic  modes. 

Manual  Augmented  Mode  -  In  the  Manual 
Augmented  mode,  the  operator  provides 
translational  and  rotational  velocity  inputs  by 
means  of  two  3  degree-of-freedom  hand  controllers 
to  command  a  selected  POR.  Feedback  displays 
to  the  operator  at  the  control  station  include  POR 
position  and  orientation  and  associated  rates,  joint 
positions  and  rates,  camera  views,  and 
representation  of  tip  forces  and  moments.  As  will 
be  discussed  later,  the  force-moment  signals  may 
be  used  in  a  closed  loop  to  provide  force-moment 
accommodation  as  an  inner  loop  to  the  operator 
control  loop. 

Single  Joint  Rate  Mode  may  be  used  by  an 
operator  to  position  one  joint  at  a  time.  The 
operator  provides  a  joint  rate  input  command  to  a 
selected  joint.  Other  joints  are  held  at  a  fixed 
position  by  the  joint  servos. 

Backup  Drive  mode  allows  joint-by-joint  motion 
using  a  separate  Back-up  Drive  Unit  (BDU).  This 
mode  is  used  to  bring  the  manipulator  to  a  safe 
state  after  the  unlikely  occurrence  of  multiple 
failures  has  disabled  primary  and  redundant 
channels  of  operation. 

Automatic  POR  Modes  -  In  the  automatic  POR 
modes,  the  POR  is  commanded  to  move  from  an 
initial  position  to  a  destination  position  in  space 
along  a  straight-line  trajectory.  The  destination 
position  may  be  specified  by  the  operator 


(Operator  Commanded  POR  mode),  or  a  series 
of  such  manoeuvres  can  be  chained  together  and 
called  up  to  form  an  automatic  sequence  of  moves 
(Prestored  POR  Autosequence  mode). 
Alternatively,  the  Artificial  Vision  Function  (AVF) 
can  provide  real-time  target  data  for  tracking  of  a 
stationary  or  moving  visual  target. 

AVF  Supported  Tracking  mode  -  The  AVF 
provides  a  significant  advance  in  the  automation  of 
space  manipulators  and  cameras  relative  to  the 
Shuttle  manipulator.  Figure  13  is  a  block  diagram 
of  the  AVF  Supported  Tracking  mode. 

A  key  element  of  the  AVF  is  the  Artificial  Vision 
Unit  (AVU)  which  processes  the  images  from  MSS 
or  Space  Station  mounted  cameras  in  real  time  and 
outputs  positions  orientations  and  velocities  of 
target  patterns  with  respect  to  the  camera,  or  with 
respect  to  a  reference  target  in  the  field-of-view. 
The  outputs  can  be  displayed  to  the  operator,  used 
to  track  targets  with  cameras  with  pan  and  tilt 
capability,  or  to  provide  feedback  to  control  the 
SSRMS  or  SPDM  manipulators  in  an  AVF 
Supported  Tracking  mode.  While  tracking  a  target 
in  this  mode,  operator  inputs  from  hand  controllers, 
or  stored  data  can  be  used  to  change  the  camera- 
to-target  tracking  distance  setpoint  in  real  time. 
This  feature  is  useful  in  operations  such  as 
automatic  berthing. 

The  feasibility  of  automated  tracking  of  targets  and 
berthing  of  payloads  using  the  AVF  has  been 
demonstrated  by  simulation  for  the  full  range  of 
SSRMS  payloads  including  the  116,000  Kg 
payload  representative  of  a  loaded  Shuttle  orbiter. 
A  simulation  response  is  shown  in  Figure  14 
where  a  22,900  Kg  payload,  representative  of  a 
Habitation  module,  is  manoeuvred  from  offset 
initial  conditions  to  a  position  and  orientation 
within  the  capture  range  of  the  berthing  mechanism 
(7.6  cm  /  1.5  deg).  For  the  simulation  it  is  assumed 
that  the  berthing  mechanism  is  equipped  with  a 
boresight  camera,  and  that  the  AVF  target  on  the 
payload  is  in  its  field-of-view  and  lock-on  has  been 
achieved.  Initial  offsets  in  translation  are 
approximately  0.8  metre  in  x,  along  the  camera 
boresight  and  about  0.2  metres  normal  to  the 
boresight.  Angular  offsets  of  the  target  axes  with 
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respect  to  the  boresight  is  approximately  9  degrees, 
with  a  roll  offset  of  5.5  degrees.  Prior  to  the 
automatic  berthing  operation,  the  module  is 
assumed  to  have  been  manoeuvred  to  the  vicinity 
of  the  berthing  mechanism  with  these  offset  initial 
conditions.  Perfect  alignment  is  achieved  when  all 
of  the  offsets  have  been  nulled.  The  SSRMS  is 
modelled  as  a  chain  of  flexible  bodies  and  includes 
the  dynamics  of  the  servomechanisms  and 
drivetrains  in  the  joints.  The  payload  is  modelled 
as  a  rigid  body,  and  the  AVU  and  camera  are 
modelled  as  an  ideal  sensor  with  a  processing 
delay. 

The  simulation  response  shows  that  the 
translational  and  rotational  displacements  converge 
smoothly  to  within  the  capture  range  in 
approximately  250  seconds  and  converge  to  very 
small  values  shortly  thereafter.  The  time  required 
to  achieve  the  berthing  mechanism  capture 
conditions  is  comparable  with  timelines  for  manual 
berthing. 

Joint  Automatic  modes  -  The  joint  automatic 
modes  are  analogous  to  the  POR  auto  modes.  In 
the  Operator  Commanded  Joint  Position  mode, 
the  operator  specifies  a  destination  set  of  joint 
positions.  The  joints  are  commanded  to  move 
from  their  current  positions  to  the  destination 
positions.  In  the  Prestored  Joint  Position  Auto 
Sequence  mode  a  series  of  such  joint  manoeuvres 
can  be  chained  together  and  called  up  to  form  an 
automatic  sequence  of  joint  moves.  Tire  joint 
automatic  modes  are  particularly  useful  in  setting 
up  specific  arm  poses  in  preparation  for  other 
operations,  or  to  deploy  an  arm  from  a  stowed 
position. 

Control  Features 

A  number  of  control  features  may  be  selected  by 
the  MSS  operator  to  enhance  manipulator 
capabilities  and  reduce  the  operator  workload.  The 
features  are  made  selectable  because  they  are  often 
useful  only  for  a  specific  type  of  operation,  or  a 
specific  manoeuvre  within  an  operation.  A  list  of 
the  selectable  features  and  a  brief  description  of 
each  is  given  in  Table  2.  The  function  of  some  of 


TABLE  2.  MSS  CONTROL  FEATURES 


FEATURE 

DESCRIPTION 

Force-Moment  Accommodation 
(FMA) 

Provides  backdriving  in  response  to 
measured  external  forces  and  moments. 

Line  Tracking 

In  POR  Auto  Modes,  controls  POR  position 
and  orientation  to  the  commanded  ’straight- 
line"  trajectories. 

Rate  Hold  Selection 

Holds  rates  commanded  at  the  time  of 
selection,  in  Manual  modes. 

Rate  Limit  Selection 

Allows  selection  of  rate  limits  within  the 
maximum  for  a  particular  payload. 

Position/Orientation  Hold  Selection 
(POHS) 

Controls  uncommanded  degrees-of-freedom 
in  Manual  Augmented  Mode. 

Rate  Input  Scale  Selection 

Allows  selection  of  rate  command  scale 
between  vernier  and  course  rates. 

Coordinate  Re-referencing 

Allows  selection  from  a  variety  of  command 
and  display  coordinate  systems. 

Manual  Mode  Trajectory  Processing 

Allows  storage  of  manual  mode  commands 
for  later  off-line  processing. 

Self  Collision  Avoidance 

Algorithm  which  prevents  collisions  between 
MSS  elements. 

Trajectory  Pause/Resume 

Allows  a  pause  and  resumption  of  motion  in 
Automatic  modes. 

Arm  Pitch  Plane  Change  (APPC) 

Commands  arm  plane  rotation  while 
keeping  the  POR  stationary. 

the  features  is  simple  and  clear  from  the  brief 
description  in  the  table.  Some  of  the  more 
advanced  and  complex  features  warrant  additional 
discussion. 

Force-Moment  Accommodation  (FMA)  -  During 
constrained  motion  operations  where  the 
manipulator  or  its  payload  come  into  contact  with 
other  structure,  some  means  of  limiting  forces  and 
moments  at  the  contact  interface  is  necessary  to 
prevent  jamming  and  possible  damage.  Force- 
moment  accommodation  provides  an  active 
compliance  characteristic  which  acts  to  relieve  the 
contact  forces  and  moments. 

A  simplified  block  diagram  describing  the  force- 
moment  accommodation  control  loop  is  shown  in 
Figure  15.  Tire  complex,  6  degree-of-freedom 
contact  dynamics  is  represented  symbolically  by  a 
simple  single  degree-of-freedom  "barrier".  The 
combined  stiffness  of  the  manipulator  and  "barrier" 
acts  as  a  gain  factor  in  the  feedback  loop  and  can 
therefore  give  rise  to  dynamic  stability  problems. 

When  FMA  is  selected,  the  manipulator  is 
commanded  in  the  normal  way  in  Manual  or 
automatic  POR  modes.  However,  signals 
representing  the  3  force  and  3  moment  components 
derived  from  the  force-moment  sensor  near  the  tip 
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of  the  arm  are  fed  back  to  modify  the  POR  rate 
commands.  This  provides  an  active  compliance  in 
the  directions  of  the  applied  forces  and  moments. 
Nonlinear  processing  in  the  feedback  loop  allows 
the  setting  of  force-moment  limits  in  real-time 
from  the  operator  control  console.  Because  of  the 
dynamic  stability  problems  referred  to  above,  the 
FMA  control  loop  necessarily  has  a  relatively  low 
bandwidth  and  is  therefore  not  effective  in 
controlling  impact  loads. 

Figure  16  shows  the  results  of  a  laboratory  test 


demonstrating  the  effectiveness  of  FMA.  In  the 
test,  the  experimental  robot  gasps  a  fixed  interface 
with  an  initial  1.3  cm  lateral  misalignment.  The 
end  effector  forces  and  moments  required  to  align 
the  end  effector  to  the  fixed  interface  are  shown 
for  the  case  without  FMA  and  then  with  FMA. 
The  action  of  the  FMA  in  relieving  contact  forces 
is  clearly  evident.  The  effect  of  the  low  bandwidth 
of  the  FMA  controller  is  demonstrated  by  a  spike 
in  the  response  during  a  rapid  change  in  load 
(impact). 


Forces/Moments 
at  Barrier 


Figure  15.  Force-Moment  Acommodation  Simplified  Control  Loop 
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Figure  16.  Force-Moment  Accommodation  Laboratory  Test  Response 
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Line  Tracking  may  be  selected  in  POR  auto 
modes  to  control  the  deviations  of  the  POR  from 
commanded  "straight  line"  trajectories  in 
translation  and  orientation.  During  an  automatic 
manoeuvre,  dynamic  transients  and  disturbances 
can  cause  the  response  of  the  POR  to  deviate  from 
the  commanded  trajectory.  If  Line  Tracking  is  not 
selected,  these  deviations  from  the  original 
commanded  trajectory  are  not  corrected.  Instead, 
at  each  control  commutation  cycle,  the  POR  is 
commanded  to  continue  along  a  new  straight  line 
from  its  present  off-track  position  toward  the 
destination.  While  the  destination  may  still  be 
achieved,  some  operations  require  manoeuvres 
down  narrow  corridors  to  the  destination  to  avoid 
collisions.  With  Line  Tracking  selected, 
deviations  from  the  original  trajectory  are 
calculated  and  fed  back  to  control  the  POR  to  the 
original  trajectory. 

Position/Orientation  Hold  Selection  (POHS)  - 

POHS  may  be  selected  in  Manual  Augmented 
mode  to  automatically  correct  deviations  due  to 
disturbances  in  uncommanded  POR  degrees-of- 
freedom,  thus  relieving  the  operator  from  manually 
correcting  these  errors  while  performing  a 
manoeuvre.  One  of  two  options  may  be  selected: 
"Hot-stick"  or  "Manual"  POHS.  When  "hot-stick" 
is  selected,  the  operator  inputs  hand  controller 
POR  rate  commands  in  the  normal  way.  However, 
at  each  control  computation  cycle,  translational  and 
rotational  vectors  are  generated  from  the  hand 
controller  signal  which  act  as  reference  trajectories. 
Translational  and  rotational  deviations  in  response 
normal  to  the  vectors  are  automatically  corrected. 
The  result  is  that  motion  in  uncommanded 
directions  is  automatically  corrected,  leaving  the 
operator  to  concentrate  on  motion  in  the 
commanded  degrees-of-freedom.  "Manual"  POHS 
is  similar  except  that  rate  command  signals  from 
selected  hand  controller  degrees-of-freedom  are  set 
to  zero.  This  option  is  useful  when  a  manoeuvre 
is  carried  out  along  a  specific  hand  controller  axis 
or  set  of  axes.  Uncommanded  axes  are  blocked 
out  to  avoid  inadvertent  command  inputs. 

Self  Collision  Avoidance  -  The  SSRMS  and 
SPDM  differ  in  the  area  of  collision  avoidance. 


The  SSRMS  uses  joint  angle  measurements  to  stop 
motion  when  a  self  collision  or  collision  with  the 
MBS  or  its  equipment  is  imminent.  The  SPDM 
provides  automated  collision  avoidance  between 
each  of  its  manipulators,  its  body  and  attached 
payload.  Where  possible,  the  collision  avoidance 
feature  uses  the  kinematic  redundancy  of  the  active 
arm  to  avoid  collisions  without  affecting  the  tip 
trajectory.  If  the  collision  cannot  be  avoided  in 
this  way,  the  trajectory  is  modified  to  avoid  the 
collision. 

Arm  Pitch  Plane  Change  (APPC)  -  It  is 

sometimes  necessary  or  convenient  to  rotate  the 
arm  pitch  plane  independent  of  payload  motion  to 
avoid  collisions,  to  allow  better  viewing,  or  to 
avoid  joint  limits.  The  arm  pitch  plane  is  the  plane 
defined  by  the  origins  of  the  three  consecutive 
pitch  joint  reference  frames  of  the  SSRMS  or 
SPDM  arms.  The  APPC  feature  allows  the 
operator  to  use  a  hand  controller  input  to  rotate  the 
arm  pitch  plane  about  the  line  joining  the  origins 
of  the  shoulder  pitch  and  wrist  pitch  reference 
frames.  The  kinematic  redundancy  of  the  SSRMS 
and  SPDM  manipulators  allows  this  pitch  plane 
motion  while  the  tip  of  the  manipulator  is  held 
stationary. 

8.0  SIMULATIONS 

Simulations  are  a  critical  tool  for  the  development 
and  operation  of  space  robotic  systems.  Due  to  the 
design  of  space  manipulators  for  operation  in  a 
weightless  environment  and  the  attendant  very  high 
ratio  of  payload  capacity  to  manipulator  mass,  the 
construction  of  hardware  models  which  are  fully 
functional  in  a  1-g  environment  is  very  difficult  or 
impossible,  as  in  the  case  of  the  SSRMS. 

High-fidelity  real-time  and  non-real-time 
simulations  are  required  in  the  development  phase 
for  feasibility  studies,  requirements  development, 
the  design  of  manipulator  control  systems,  control 
modes  and  the  human-machine  interface,  and  the 
verification  of  the  overall  human-machine  system 
design  as  well  as  the  manipulator  handling 
performance  before  flight.  For  space  robotic 
operations,  simulations  are  used  for  operations 
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planning  and  support,  operator  training  and 
sustaining  engineering. 

In  the  development  of  simulations  for  the  MSS 
particular  challenges  arise  due  to  the  need  to 
simulate  high-order  dynamic  systems  with  multiple 
elastic  bodies  which  can  be  connected  in  different 
topologies  such  as  chains,  trees,  loops  and  nests. 
The  need  to  simulate  the  transition  from  one 
topology  to  another,  e.g.  during  an  ORU  change- 
out  operation,  and  to  represent  the  dynamics  of 
contact  during  such  transitions  pushes  the  state-of- 
the-art  in  dynamic  modelling  techniques  and  tools. 

9.0  CONCLUSIONS 

Robotics  will  play  an  increasingly  critical  role  in 
the  hostile  space  environment  as  more  and  more 
complex  tasks  are  taken  on  in  the  assembly  and 
maintenance  of  space  systems.  The  MSS  fulfils 
the  requirements  for  the  current  robotic  operations 
on  the  international  Space  Station.  Many  of  the 
requirements  evolve  from  tasks  which  are  unlike 
any  which  have  previously  been  assigned  to  space 
manipulator  systems.  Substantial  progress  has 
been  made  in  the  design  of  systems  which  meet 
very  stringent  control  and  operational  capabilities 
and  perfonn  a  wide  range  of  tasks  in  space.  To 
accomplish  this,  a  variety  of  modes  of  operation 
and  selectable  control  features  have  been  designed 
into  the  systems  for  human-in-the-loop  and 
automatic  operation.  These  serve  to  broaden  the 
scope  of  operating  capabilities  of  the  manipulators 
and  increase  their  productivity,  along  with  that  of 
the  human  operators.  In  the  future,  even  more 
sophisticated  space  manipulator  systems  will  be 
developed  as  current  research  in  robotics  is  applied 
to  real  space  hardware. 
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SUMMARY 

The  French  space  Agency  (CNES)  started  a  study  in  late 
1992,  of  an  autonomous  rover  VAP  (for  Planetary 
Autonomous  Vehicle).  The  aim  of  this  study  was  to 
investigate  multimission  mobile  platform  design.  The  focus 
was  placed  on  a  martian  mission  for  several  reasons: 

-  there  is  a  very  high  scientific  interest  for  Mars  surface 
exploration  leading  to  a  better  understanding  of  the  solar 
system  and  Earth  evolution, 

-  roving  on  the  planet  is  one  mandatory  and  preliminary 
step  before  the  conquest  of  the  "red  planet"  by  manned 
mission, 

-  it  is  a  necessary  complement  to  fixed  networks  and 
sample  return,  in  order  to  get  data  relevant  to  very  large 
areas. 

The  overall  system  concept  including  launch,  cruise, 
deboost  from  Mars  orbit,  Mars  atmosphere  entry  and 
landing  is  not  part  of  the  study  but  is  only  kept  in  mind,  as 
these  phases  of  the  mission  induce  several  constraints. 

The  main  results  of  the  study  are  given,  showing  the  two 
possibilities: 

-  a  large  vehicle  of  450  kg  as  the  baseline. 

-  a  smaller  vehicle  of  250  kg  as  an  option. 

The  various  sub-systems  are  described  and  the  choices 
justified.  The  expected  performances  are  summarised. 

1-  THE  REQUIREMENTS 

The  goal  is  to  design  a  rover  providing  the  maximum 
capability  for  science,  devoted  to  Mars  exploration. 

The  overall  system  is  not  part  of  the  study,  but  the  rover 
must  be  compatible  with  an  ARIANE  5  Launch  and  survive 
the  long  cruise  to  the  planet.  As  well,  the  rover  must 
interface  with  a  descent  module  and  an  orbiter  during  this 
phase  toward  Mars. 

We  have  considered  the  rover  is  arrived  on  the  martian 
surface.  Then  it  must  have  the  following  mission 
capabilities:  travel  500  to  1000  km.  with  a  rather  good 
knowledge  of  its  position  (5  to  10%).  This  travel  is  expected 
to  take  place  in  between  two  dust  storms  (450  days  max.) 
with  possible  extension  after  the  wind  period. 

In  detail,  the  VAP  must  be  able  to  fulfil  three  different 
missions: 

-  lay  down  and  install  three  small  fixed  scientific  stations 
at  three  different  predefined  locations  on  Mars, 

-  perform  geophysical  measurements  during  the  move  on 
the  planet  surface, 

-  travel  as  an  autonomous  laboratory,  able  to  analyse  the 
environment,  ground  samples,  etc... 


The  vehicle  must  of  course  withstand  the  martian 
environmental  conditions  attached  to  the  expected  landing 
site  in  the  "tempered"  region  of  the  northern  hemisphere: 
external  temperatures  between  -50°C  and  +30°C,  low 
gravity  (1/3  Earth  g.),  thin  C02  atmosphere  (less  than  1% 
Earth  density).  It  must  also  be  able  to  rove  on  an  unknown 
surface  made  of  sand  dunes,  quicksands,  or  hard  lava,  rocks, 
boulders,  pebbles  etc...,  climb  slopes  of  24°  and  overpass 
obstacles  0.5  m  height.  The  complete  machine  will  be 
sterilized  to  avoid  any  pollution  or  contamination  of  the 
planet. 

Finally  VAP  must  send  down  to  Earth  the  scientific 
measurements  results,  via  an  orbiter  link  (high  data  rate) 
or  directly  to  Earth  (house  keeping  data). 

2-  THE  INDUSTRIAL  TEAM  AND  BACKGROUND 

One  of  the  characteristics  of  these  new  robotic  systems  is 
the  wide  collection  of  techniques  involved  which  implies  to 
set  up  a  complete  industrial  team  having  all  the  various  and 
necessary  background. 

For  this  reason,  the  organization  of  the  study  was  led  by 
AEROSPATIALE,  which  has  the  necessary  background 
and  skill  for  mechanical  structure,  thermal  aspect, 
integration  and  system  aspect,  as  usual  for  spacecraft;  but 
also  which  has  the  specific  knowledge  for  atmosphere  entry 
shieldings  and  protection  shells. 

The  overall  electrical  system  is  under  the  responsibility  of 
ALCATEL  ESPACE  which  has  the  necessary  background 
and  skill  for  power  supply,  telecommunication,  on-board 
data  processing,  and  robotic.  In  this  last  technique, 
ALCATEL  ESPACE  had  a  strong  support  from  the 
corporate  research  center:  ALCATEL  ALSTFIOM 
RESEARCH  which  has  spent  more  than  10  M$  on  the 
subject  along  with  FRAMATOME  during  the  last  four 
years,  leading  to  two  demonstration  models  of  rovers: 

-  one  devoted  to  locomotion, 

-  one  devoted  to  autonomous  navigation. 

Finally,  the  robotic  activity  is  completed  by  a  third  partner: 
SAGEM  specialized  in  navigation,  localization  and  sensors 
for  both  applications  (space  and  vehicles).  Their 
background  and  skill  is  also  articulated  around  a 
demonstration  model,  built  in  a  4  wheel  drive  vehicle. 

3-  METHOD  OF  INVESTIGATION 

Two  different  sizes  of  vehicle  have  been  investigated: 

-  a  large  vehicle  of  450  kg.  including  125  kg.  of  payload 
equipments,  able  to  rove  1000  km  during  600  days  and 
to  locate  itself, 

-  a  smaller  vehicle  of  250  kg.  including  75  kg.  of  payload 
equipments,  with  the  same  capability  if  possible. 
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Looking  at  the  requirements,  it  is  to  be  noticed  that  VAP 
is  several  things  at  the  same  time: 

-  it  is  a  spacecraft  first, 

-  it  is  also  a  robot,  which  must  be  able  to  work  alone, 

-  it  is  finally  a  vehicle  (like  a  car)  with  improved  roving 
capabilities. 

The  best  compromise  is  to  be  found  in  between  these 
functions.  For  this  reason,  a  permanent  iterative  process  is 
used,  raising  new  constraints  on  the  subsystems. 

Therefore,  we  think  that  the  output  of  the  study  is  an 
adaptative  design  minimizing  the  modifications  necessary 
for  exploring  different  areas,  or  other  planets  like  the  moon. 

As  a  consequence  of  the  VAP  multiple  functions,  the 
architecture  study,  traditional  for  ordinary  satellites, 
follows  a  different  approach  in  that  case,  because  of  the 
displacement  of  the  vehicle  : 

-  the  structure  must  withstand  the  launching  constraints 
and  also  those  generated  by  the  displacement  which 
induce  mechanical  fatigue, 

-  the  autonomy  must  be  managed  on  a  day  by  day  basis, 
and  compromises  are  to  be  done  between  the  vision,  the 
locomotion,  and  the  constraints  coming  from  the  other 
subsystem  as  the  power  supply, 

-  mixing  of  traditional  subsystem  with  robotic  subsystems 
implies  a  real  mastering  of  all  the  needs  in  order  to 
optimize  the  on-board  data  management  resources, 

-  the  degraded  modes  are  studied  from  the  very  beginning, 
as  well  as  the  safety  aspects,  because  their  impacts  on  the 
general  architecture  could  be  important. 

It  is  not  a  satellite  which  moves,  but  a  vehicle  which  is  to 
be  launched  toward  an  unknown  planet. 

Therefore,  the  platform  can  be  split  in  two  different  types 
of  subsystems : 

-  classical  subsystems  (as  for  satellites), 

-  robotic  subsystems. 

The  classical  subsystems  are  : 

-  mechanical  and  mechanisms, 

-  thermal, 

-  telecommunication, 

-  power, 

-  on-board  data  processing. 

The  robotic  subsystems  are  : 

-  perception/navigation, 

-  autonomous  path  generation, 

-  autonomous  localization. 

The  study  analyses  each  subsystem  needs  in  term  of  energy 
and  data  handling  capability,  in  order  to  choose  an 
on-board  architecture  which  best  use  the  available 
capability,  by  means  of  specialized  parts. 

During  the  study  several  iterations  will  be  made  to 
permanently  look  for  a  compromise  between  the 
subsystems  in  order  to  find  the  best  optimum  performances 
for  the  vehicle. 

A  choice  for  the  data  management  subsystem  (DMS)  is 
done  from  the  functional  study  analysis  of  the  needs,  in 
order  to  optimize  the  processing  resources.  A  centralized 
processing  is  used  as  a  starting  point  for  this  analysis. 

4-  THE  PAYLOAD 

No  specific  work  was  made  on  the  payload  equipments. 
They  are  arranged  in  a  way  to  meet  the  available  place  and 
to  cope  with  the  other  aspects  as  thermal,  power  supply, 
volume,  weight. 

Only  few  characteristics  or  aspects  are  taken  into  account 
from  the  payload: 

-  One  concerns  the  3  fixed  stations  to  be  dropped.  They 
are  mounted  in  locations  where  the  mechanical  arm  can 
reach  them. 


-  Another  concerns  the  interaction  possibilities  with  the 
autonomy  of  movement  of  the  vehicle.  In  other  word,  the 
following  idea  has  been  investigated:  it  must  be  very 
interesting  to  cooperate  with  the  payload  in  order  to 
detect  gradients  for  example.  As  a  consequence,  such 
detection  made  autonomously,  may  change  the  mission 
of  the  day,  to  follow  the  gradient  (as  water  detection  for 
example). 

Such  payload/system  cooperation  has  been  considered  as 
necessary  by  the  scientists  in  order  to  improve  the  capability 
of  founding  water  on  Mars,  which  is  one  of  the  main 
concern. 

5-  THE  VARIOUS  SUB  SYSTEMS 

The  platform  can  be  split  in  several  subsystems: 
the  mechanical  chassis  with  the  wheels 
the  telecommunication 
the  data  management 
the  robotic,  including: 
mission  supervision 
autonomous  path  generation 
locomotion 

localization  and  navigation 
perception 
the  power  supply 
the  thermal  control 

5.1-  THE  CHASSIS 

The  design  of  the  chassis  must  meet  several  requirements 
which  are  opposite  from  each  other: 

When  folded,  the  volume  must  be  minimum,  and  when 
unfolded,  the  centre  of  gravity  must  be  as  low  as  possible. 
The  capability  in  various  terrains  depend  mainly  on  the 
shape  of  the  wheels: 

-  If  they  are  large  it  is  better  for  sand  dunes, 

-  if  they  are  narrow,  it  is  better  for  the  total  vehicle  volume 
when  folded. 

Finally  in  order  to  get  out  from  difficulties  as  for  example 
in  quicksands,  the  chassis  must  provide  a  "crawling" 
capability  in  association  with  the  wheels. 

Two  different  chassis  have  been  designed,  one  for  the  large 
VAP,  based  on  a  4  wheel  chassis.  The  second  one  for  the 
small  VAP  has  6  wheels  and  is  rather  similar  to  the  Russian 
Marsokod  from  the  "Mars  96"  project.  Both  of  them  have 
the  crawling  capability  (see  the  fig.). 
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Small  YAP 


The  large  chassis  can  orientate  its  4  wheels  in  order  to  turn 
in  place  or  to  stop  moving  in  a  slope  (brake).  Longitudinal 
sliding  rods  allow  the  crawling.  A  longer  vehicle  can  be 
obtained  by  this  way  which  vary  the  relative  weight  applied 
on.  each  pair  of  wheels.  The  front  axle  is  passively 
articulated  to  the  main  frame.  The  wheels  are  nearly  1  m. 
in  diameter. 

The  small  chassis,  can  also  be  lengthen  for  crawling.  The  3 
axles  are  passively  articulated  to  each  other.  The  main 
characteristics  of  this  chassis  are  the  3  separated  platforms 
available  for  the  equipments.  This  small  chassis  is  quite 
efficient  on  various  terrains:  it  overpasses  obstacles  as  high 
as  its  wheel  diameter. 

5.2-  THE  TELECOMMUNICATION 

This  subsystem  has  five  different  missions  to  fulfil: 

-  provide  a  link  directly  to  Earth  for  teleprogramming  of 
the  rover  and  tranfering  operational  data.  400  Kbits  per 
day  up  link  and  2  Mbits  per  day  down  link, 

-  provide  a  link  via  a  relay  satellite  orbiting  Mars  for  the 
scientific  data.  12  Mbits  per  day  down  link  plus  the  direct 
link  capability  for  back-up  purpose, 

-  provide  a  back-up  mode  with  lower  channel  capability, 

-  provide  a  permanent  standby  mode  for  telecommand 
access, 

-  allow  to  locate  the  vehicle  by  mean  of  the  radiometric 
measurements  between  the  rover,  the  Earth  and/or  the 
orbiter. 

The  ground  space  station  uses  nominally  the  34  m  diameter 
antenna  from  the  DSN  (Deep  Space  Network),  the  70  m 
antenna  can  be  used  only  as  a  back-up.  The  use  of  the  station 
is  limited  to  1  hour  each  time  (2  hours  a  day). 

The  quality  of  the  link  is  such  that  the  frame  error  rate  is 
lower  than  10"'  for  99%  of  the  time. 

An  hemispherical  antenna  coverage  is  enough  for  all  the 
links  from  the  vehicle. 

The  chosen  frequency  band  is  X  band  which  corresponds 
to  the  best  optimum. 

S  band  has  been  rejected  because  of  the  too  large  size  of 
the  antenna  and  also  because  of  the  difficulties  for  VLBI 
measurements  for  localization  (Very  Large  Base 
Interferometry). 

Ka  band,  on  the  other  side,  does  not  give  smaller  antenna 
compared  to  Xband,  because  of  the  link  availability 
requested.  Moreover  S  or  X  band  is  nevertheless  necessary 
as  a  back-up  to  the  Ka  link  which  should  lead  to  a  mixt 
complex  solution. 


Technology  of  the  TELECOMMUNICATION 

For  the  X  band  direct  link  antenna,  the  chosen  solution  is 
to  have  a  flat  antenna  electronically  steered  in  elevation, 
and  mechanically  oriented  for  azimuth.  The  antenna  is 
feeded  by  40W  SSPA  (Solid  State  Power  Amplifiers). 

For  the  relay  link,  the  same  kind  of  antenna  is  chosen. 

Both  antenna  will  be  mounted  on  the  same  flat  support  (see 
the  fig.),  r  ' 
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MGA-TX:  15  d8! 


MGA-RX:  9dOI 


HGA-TX:  29  dBI 


HGA-RX:  20  dBI 


Large  YAP  flat  pannel  antenna 
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MGA-TX:  19  dBI 
(cone  a  3  dB) 

MGA-RX:  14  dBI 
(cone  a  3  dB) 


Small  YAP  flat  pannel  antenna 


Low  gain  antenna  are  also  provided  for  standby  mode  and 
back-up  mode. 

The  remaining  equipment  of  the  telecommunication 
subsystem  are  two  X  band  coherent  transponders,  a  RF 
distribution  unit,  two  SSPA. 

The  total  mass  is  around  23  kg  for  the  large  VAP  for  135 
Watts  peak  during  communication  (10W  standby). 

For  the  small  VAP  the  subsystem  can  be  simplified  when 
considering  a  lower  channel  capability  for  the  direct  link. 
The  SSPA  are  only  10  Watts  each. 

The  total  weight  becomes  14  kg  for  50  Watts  peak  (10W 
standby). 

The  procedures  of  communications  are  in  accordance  with 
the  ESA  standards. 

5.3-  THE  ON-BOARD  DATA  MANAGEMENT  (DMS) 

The  functions  of  this  subsystem  is  to  ensure: 

-  the  autonomy  of  the  vehicle  in  martian  environmental 
conditions, 

-  the  working  of  the  vehicle  and  the  payload, 

-  the  safety  when  working  and  the  capability  to  get 
information  back. 

The  subsystem  manages  the  communications.  For  this 
purpose,  it  controls  the  direct  link  and  the  relay  link.  The 
communications  are  made  on  appointment  (or  rendezvous) 
every  day. 
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This  implies: 

-  the  pointing  of  the  antenna  according  to  the  ephemens, 

-  the  set  up  of  back  up  link  if  necessary, 

-  the  data  processing  necessary  prior  to  start  the 
transmission  (multiplexing,  format  set  up,  storage,  etc...). 

The  DMS  performs  also  the  mission  control.  For  that 
purpose,  it  keeps  the  on-board  time,  manages  the  high  level 
modes  of  the  rover  like: 

-  starting, 

-  system  configuration, 

-  mission, 

-  failure  management, 

-  emergency  modes, 

in  accordance  with  the  various  operational  constraints  and 
the  various  phases  during  the  lifetime  of  the  vehicle. 

The  DMS  manages  also: 

-  the  lower  mission  modes  according  to  the  daily  mission 
plan  loaded  every  morning, 

-  the  on-board  time, 

-  the  vehicle  status,  and 

-  the  tasks. 

The  DMS  controls  the  execution  of  the  tasks,  by  reading 
the  associated  scripts,  and  sends  all  the  necessary 
commands  for  that. 

Another  important  function  of  the  DMS  is  to  manage  the 
on-board  resources  for  the  other  subsystems  : 

-  ON-OFF  commands, 

-  share  the  available  computer  time  and  memory. 

It  is  also  very  important  to  know  the  status  of  the  vehicle. 
This  is  the  reason  why,  special  attention  is  placed  on  this 
function,  performed  also  by  the  DMS.  For  this  purpose, 
several  formats  are  sent  back  to  Earth  : 

-  real  time  housekeeping  telemetry, 

-  reporting  telemetry  (from  memory), 

-  upon  request  from  ground:  failure  investigation 
telemetry. 

The  working  of  the  VAP  is  checked  at  two  different  levels: 

-  A  specific  function  called  "supervision"  checks  that  the 
commands  are  correctly  executed  by  looking  at  the 
equipment  status  (monitoring  information).  In  case  of 
trouble  it  turns  the  system  to  failure  management  mode. 

-  An  independent  function  permanently  checks  the  direct 
housekeeping  telemetry  like  temperatures,  voltages, 
B.I.T.E.  Status  (Built-In  Test  Equipment),  etc...  This 
function  turns  the  system  to  reconfiguration,  if  necessary. 

At  the  system  level,  several  constraints  applied  on  the 
vehicle  are  identified.  These  constraints  were  driving 
factors  for  the  main  design  choice:  The  duration  of  a 
transmission  from  Earth  to  Mars  can  be  as  long  as  20 
minutes  one  way  (40  min.  for  the  answer)  which  correspond 
to  the  greatest  distance  (worst  case).  This  implies  a  large 
autonomy  of  the  vehicle  during  at  least  one  day.  Therefore 
the  commands  are  high  level  commands  in  order  to 
minimize  the  transmission  time.  On  the  other  way,  the 
telemetry  data,  stored  during  one  day,  are  transmitted 
according  to  their  relative  priority. 

Technology  of  the  DMS 

The  architecture  study,  has  shown  that  the  data 
management  attached  to  the  mission  can  be  separated  in 
two  categories: 

-  the  processing  relative  to  the  vehicle  management  and 
mission  control, 

-  the  processing  needed  for  vision,  and  moving  of  the  rover 
(path  generation  and  control,  stereovision  processing). 

The  first  category  corresponds  to  a  permanent  work  of  the 
rover,  and  needs  a  medium  computing  power  (2  Mips).  The 
other,  at  the  contrary,  is  working  only  during  the  movement 
of  the  rover  and  needs  a  high  computing  power 
(>_  10  Mips). 


Therefore,  two  different  computers  are  implemented.  The 
moving  processing  is  chosen  to  run  on  a  RISC  computer 
(10  Mips). 

The  remaining  processing  will  run  on  a  MIL-STD-1750 
processor  (1.3  Mips). 

Because  of  the  small  communication  channel  capacity,  the 
operational  softwares  will  be  implemented  before  launch. 
The  non  volatile  memories  will  be  EEPROM  and  bubble 
memory. 

The  overall  DMS  sub-system  is  evaluated  to  weight  19  kg 
including  5  kg  of  redundancy.  The  power  consumption  is 
35 W.  and  can  be  reduced  to  25W.  with  a  limited 
redundancy. 

For  the  small  VAP,  some  limitation  in  the  DMS  capabilities 
are  envisaged  which  allows  to  go  down  to  11  kg.  and  12W. 

5.4-  THE  ROBOTIC 

The  robotic  functions  must  ensure  the  missions  of  the 
planetary  rover  and  its  move. 

The  main  robotic  functions  are: 

-  the  mission  supervision  determines  the  tasks  to  perform, 
how  to  do,  and  manages  the  overall  functions  of  the 

vehicle,  .  . 

-  the  autonomous  path  generation  determines  in  advance 
which  way  to  take,  with  only  sensors  data  as  inputs, 

-  the  movement  control  executes  the  determined 
movement. 

In  addition,  other  peripheral  functions  must  be  added  as : 

-  the  sensors,  which  give  data  according  to  the  vehicle 
environment  or  surrounding, 

-  the  localization,  which  delivers  data  concerning  the 
position  of  the  vehicle  on  the  planet,  and  also  attitude 
information. 

Some  requirements  placed  on  the  robotic  are  coming  from 
the  mission :  „„„  ,  .  ... 

-  The  total  distance  to  travel  (500  to  1000  km)  which 

imposes  to  drive  2.5  km  per  day  with  only  5  hours  of 
movement  in  one  day.  These  figures  imply  to  make  10 
meters  steps  with  a  maximum  processing  time  of  1  minute 
per  step.  . 

-  Some  of  the  payload  tasks,  can  have  an  impact  on  the 
robotic  functions,  in  term  of  time,  precision,  etc... 

-  The  VAP  must  be  flexible  enough  to  change  the  overall 
mission  if  the  first  scientific  results  say  so.  In  addition, 
some  interaction  can  occur  (ex. :  to  follow  a  gradient  seen 
by  the  payload). 

-  The  martian  environment  is  quite  unknown.  Only  a  map 
with  10  meter  resolution  is  available. 

-  VAP  must  be  autonomous  as  already  seen,  because  of 
the  transmission  delays. 

-  Finally,  VAP  must  be  relailable  enough  to  run  correctly 
during  at  least  1  year  (400  days):  neither  failure  nor 
accident. 

Some  requirements  are  coming  from  other  subsystems: 

-  Mass  and  available  power  supply  are  limited  which 
imposes  to  limit  the  software  sizes,  the  computation  time 
and  speed,  the  memory  sizes,  some  actions  are  performed 
in  series,  instead  of  parallel. 

-  Communication  rates  are  also  limited.  This  prevents 
performant  loading  of  information  every  day  and  makes 
software  updates  difficult. 

-  Perception  is  limited  to  1  meters  which  limits  the  sizes  of 

the  moving  steps.  The  resolution  is  important  too,  in 
order  to  "see"  the  obstacles.  , 

-  The  agility  of  the  vehicle  is  also  important,  and  its 
capability  to  overpass  obstacles  influences  the  robotic. 
More  obstacle  can  be  overpass,  simpler  will  be  the  robotic 
processing. 
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5.4.1-  MISSION  SUPERVISION  (decisional  structure) 

This  is  the  heart  of  the  robotic,  always  in  touch  with  the 
data  management  subsystem  (DMS). 

It  must  perform  the  tasks  loaded  on-board  every  day. 

For  this  purpose,  the  mission  suspervision,  controls  all  the 
elementary  functions  as: 

-  Perception. 

-  Localization. 

-  Path  generation. 

-  Movement  control. 

-  Payload  management. 

We  have  seen  above  that  the  DMS  must  keep  VAP  "alive", 
i.e.  takes  limited  actions.  On  the  contrary,  the  mission 
supervision  allows  VAP  to  be  active,  i.e.  to  do  what  is 
requested  to  do.  DMS  has  the  priority  and  can  moderate 
the  VAP  actions  and  tasks. 

The  mission  supervision  is  centralized  in  the  VAP.  It  knows 
all  what  is  happening  and  how  is  the  VAP.  Only  actions  in 
front  of  danger  are  not  directly  activated  by  the  supervision 
but  automatically  triggered  by  the  subsystem  which  detects 
the  danger. 

Through  this  mission  supervision,  VAP  is  programmable: 

-  at  mission  level  which  defines  the  tasks  to  performs  and 
the  attached  constraints, 

-  at  the  specification  level  of  the  tasks,  with  the  "scripts" 
which  define  the  cascade  of  actions  and  processing  for  a 
given  task. 

The  mission  supervision  receives  an  error  message  in  case 
of  task  failure.  This  means  that  for  example  the  movement 
is  stopped,  or  something  wrong  appears.  The  mission 
supervision  can  decide  to  do  the  same  task,  but  in  another 
way  by  selecting  new  scripts. 

If  the  error  is  coming  from  a  hardware  or  software  failure, 
a  diagnostic  is  run  in  order  to  find  the  reason  why. 

Finally, _  the  mission  supervision  must  manage  the  time 
constraints  associated  to  the  mission  plan. 

5.4.2-  AUTONOMOUS  PATH  GENERATION 

The  path  generation  has  two  purposes: 

-  Determine  the  path  necessary  forgoing  from  one  location 
to  the  other  in  order  to  be  closer  and  closer  from  the  final 
objectives. 

-  Check  in  advance  the  "safety"  attached  to  the  generated 
path  in  order  to  determine  whether  the  move  envisaged 
is  safe  and  without  risk. 

It  is  to  be  noticed  that  pure  reactive  methods  are  not 
envisaged  for  VAP  as  they  appear  too  risky  because  in  this 
case  no  feasibility  of  the  movement  is  checked  in  advance. 
The  VAP  could  enter  dangerous  situation  with  no  return, 
which  is  highly  probable  if  the  trip  is  long.... 

Sensor  acquisition  and  movement  are  supposed  to  be  made 
sequentially.  Parallel  processing  could  have  been  retained 
but  was  finally  rejected  for  the  following  reasons: 

-  sensor  precision  is  worst  when  moving, 

-  processing  time  is  not  long, 

-  limited  available  power  supply  implies,  anyway,  to  slow 
down  the  complete  sequence. 

The  autonomous  path  generation  interfaces  with  the  other 
subsystems: 

The  mission  suspervision  first,  which  controls  the  path 
generation.  It  receives  data  also  from  the  localization 
(position  or  location,  and  attitude  fo  the  VAP)  and 
perceptions  (distance  pictures,  and  data  concerning  the 
ground  surface) 


The  autonomous  path  generation  delivers  informations  to 
the  movement  control  (path  to  follow,  locomotion  modes, 
etc...). 

It  exists  several  methods  to  generate  a  path,  which  are  more 
or  less  efficient.  The  choice  has  been  made  on  a  concept 
which  consists  in  matching  the  method  to  the  difficulty  of 
the  terrain.  This  method  is  the  quickest.  To  three  types  of 
ground  are  attached  three  different  methods  for  the  path 
generation: 

-  The  so  called  "ID  ground" 

If  there  is  just  few  small  obstacles  (pebbles)  on  a  surface 
rather  flat,  the  simplest  method  is  to  use  directly  the  data 
coming  from  the  vision  sensors.  It  is  only  necessary  to  check 
whether  to  go  straight  on  is  valid  or  not,  as  a  path  toward 
the  objective,  or  to  determine  a  direction  in  order  to  head 
for  the  objective  rapidly. 

-  The  so  called  "2D  ground" 

These  terrains  correspond  to  obstacles  like  boulders  or 
small  blocks  placed  on  a  still  rather  flat  surface.  In  these 
conditions  it  is  only  necessary  to  notice  what  is  not  "flat" 
which  corresponds  to  obstacles,  and  to  generate  the  path 
turning  around  those  obstacles,  but  without  stability  check 
(when  obstacles  are  avoided,  the  land  is  rather  flat). 

-  The  so  called  "3D  ground" 

For  these  types  of  ground,  it  is  necessary  to  use  the  general 
method  of  path  generation :  it  consists  in  modelling  the  land 
in  3D,  and  to  plan  a  path  in  a  3D  space,  i.e.  taking  into 
account  the  stability  of  the  robot  and  the  spare  distance 
below  the  robot,  for  every  configuration. 

Considering  the  expected  performance  of  the  robot 
(overpassing  capability  of  50  cm  height,  slope  of  24°),  we 
have  assumed  that: 

70%  of  ground  are  ID  type 
25%  are  of  2D  type 
5%  are  of  3D  type. 

The  selection  of  the  method  for  path  generation  according 
to  the  ground  difficulty  should  bring  an  important 
improvement  of  performances  against  one  method  chosen 
in  advance. 

The  path  generation  will  be  made  of  the  following 
sub-functions: 

-  selection  of  the  strategy  (ID,  2D  or  3D), 

-  modelling:  from  the  sensor  data,  a  flat  model  is 
elaborated  where  the  obstacles  are  placed  (2D  ground) 
or  a  digital  model  of  the  ground  is  elaborated  (3D 
ground), 

-  local  adjustment  of  the  new  model  inside  the  known  map, 

-  updating  of  the  map, 

-  management  of  the  map, 

-  intermediate  target  determination, 

-  path  generator  (simple  if  ID,  on  flat  ground  when  2D, 
on  rough  areas  when  3D), 

-  moving  plan  (determine  the  series  of  command 
movements  locomotion  mode,  parameters  to  look  at....). 

This  last  trajectory  plan  is  working  on  a  medium  scale 
representation  (the  path  generation  is  working  on  a  large 
scale  representation),  and  is  in  charge  of  finding  an  "a  priori" 
feasible  trajectory,  that  takes  into  account  the  kinematics 
and  locomotion  constraints. 

Performances  of  the  path  generation: 

Several  estimates  have  been  done,  leading  to  the  following 
expected  performances: 

processing  time  (assuming  a  SPARC  2  processor): 

-  4  sec  in  a  ID  ground, 

-  15  sec  for  2D 

-  30  sec  for  3D 
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For  the  small  VAP,  the  approach  must  be  more  "reactive" 
in  order  to  decrease  the  processing  need  (speed-power) 
during  most  of  the  moves  (70%  ID).  The  higher  levels  are 
kept  to  the  expense  of  processing  time.  The  reason  why  is 
to  keep  the  running  of  the  rover  as  safe  as  possible.  In 
addition,  the  mechanical  chassis  will  help  as  it  is  more  easy 
in  terms  of  overpassing  capabilities. 

5.4.3-  LOCOMOTION  (MOVEMENT  CONTROL) 

This  subsystem  is  in  charge  of  the  real  move  of  the  rover: 
it  uses  the  chassis  of  the  rover  (mechanical  structures, 
wheels,  and  associated  internal  sensors). 

The  chassis  generally  moves  with  its  wheels  (electrical 
motors).  It  can  move  also  using  it  variable  length  (crawling). 

This  defines  several  modes  of  locomotion,  i.e.  different 
ways  for  following  the  same  path. 

All  its  commands  are  coming  from  the  mission  supervision 
except  the  emergency  stops  which  may  come  from  another 
subsystem  in  case  of  reflex  actions. 

The  possible  commands  always  allow: 

-  to  stop  and  start  again  a  movement, 

-  to  cancel  a  movement, 

-  to  define  a  new  movement  to  do. 


The  expected  performances  are  the  following: 

-  Absolute  position  <  500m 

-  Relative  position  <  1% 

-  Relative  altitude  <  1%  of  the  move 

-  Direction  precision  <  0.3°  always 

-  Tilt  angles  <  0.3°  always 

For  the  small  VAP,  these  equipments  need  to  be  reduced 
and  the  solutions  are  slightly  different: 

-  A  solar  sensor  replaces  the  star  sensor. 

-  Only  dead  reckoning  is  used. 

-  Solar  sensor  and  gyrometers  for  the  direction. 

-  Accelerometers  and  gyrometers  for  tilt  angles. 

In  these  conditions  the  performances  will  be  11  kg  for 
30  Watts  peak : 

-  Absolute  position  <  5  km 

-  Relative  position  5  to  6% 

-  Relative  altitude  <  1%  of  the  move 

-  Direction  precision  <  0.3°  when  stopped  with  sun 

below  80° 
otherwise :  <  1° 

-  Tilt  angles  <  0.3°  always 


5.4.5-  PERCEPTION 


A  movement  is  fully  described  by: 

-  a  path  mode  with  describing  parameters, 

-  a  locomotion  mode  (to  be  used), 

-  safety  conditions. 

The  safety  conditions  are  tested  via: 

-  either  the  localization  subsystem  (stability), 

-  or  the  vision  sensors  (detection  of  obstacles), 

-  or  the  movements  and  internal  sensors  (sliding  detection, 
blocking  of  one  wheel,  etc...). 

This  subsystem  can  be  split  in  several  loops: 

-  one  loop  determines  the  moving  speed  in  accordance 
with  the  kinematics  of  the  vehicle  in  order  to  keep  it  on 
the  path, 

-  one  loop  which  directly  drives  the  motors  (wheels’speed, 
and  orientation), 

-  one  loop  which  checks  permanently  the  safety  conditions. 

5.4.4-  LOCALIZATION  AND  NAVIGATION 

The  purpose  of  this  subsystem  is  to  have  3  functions: 

-  to  give  localization  information  (where  is  the  rover  on 
Mars  ?), 

-  to  deliver  stability  of  the  rover  (tilt  angle  with  reference 
to  vertical), 

-  to  give  navigation  information  (where  is  the  North,  in 
order  to  know  which  direction  to  take). 

For  these  reasons,  this  subsystem  is  composed  of  several 
sub-units: 

*  a  star  sensor  with  vertical  reference.  It  delivers  the 
absolute  localization  of  the  VAP  on  the  planet, 

*  an  inertial  navigation  item,  which  is  reset  during  the 
VAP  steps.  It  gives  the  relative  localization  during  the 
moves, 

*  a  gyrocompass  for  initial  setting  associated  with 
gyrometers  during  the  movement.  They  gives  the 
reference  direction  in  order  to  known  where  to  head 
for, 

*  accelerometers  of  initial  setting  associated  with  the 
same  gyrometers  during  the  movement.  They  gives 
the  tilt  angles  of  the  vehicle  (pitch  and  roll)  for 
stability  purpose. 

All  these  sensors  are  mounted  on  the  structure  of  the 
vehicle.  There  is  no  stabilised  platform.  The  star  sensor  is 
oriented  45°  from  the  vertical. 

The  overall  mass  is  estimated  to  be  less  than  26  kg  for  50  W. 
peak  of  power  consumption  during  the  moves. 


The  perception  subsystem  corresponds  to  the  eyes  of  the 
VAP.  It  consist  in  the  association  of  the  sensors  with  the 
corresponding  data  processing  in  order  to  deliver 
information  which  can  be  used  by  other  subsystem. 

Several  function  must  be  performed  in  this  subsystem: 

-  The  3D  perception:  this  will  be  done  by  mean  of  a  wide 
angle  laser  range-finder.  In  addition  stereovision  is 
envisaged  as  a  back-up  solution. 

-  The  range  measurement:  it  will  be  done  with  the  laser 
range  finder.  The  back-up  is  envisaged  with  a  one-line 
laser  range-finder. 

-  Nature  of  the  soil  evaluation:  this  will  be  done  with  the 
infrared  sensors  from  the  payload. 

It  is  to  be  noticed  that  the  detailed  study  has  shown  the 
great  difficulty  to  perform  the  landmark  recognition  and 
the  horizon  profile  matching  (to  be  used  for  the  localization 
purpose). 

The  main  performances  expected  are  the  following: 

Distance:  10  m 

Field  of  views:  100  deg 

(2  times  50°  range  finder) 

Total  mass:  13  kg 

Power  consumption:  80W  peak 

(during  2  sec.  each  time) 

0.5W  standby 

For  the  small  VAP,  the  two  range-finders  will  be  replaced 
by  stereovision  for  the  3D  perception. 

The  range  measurement  will  be  made  by  a  small  distance 
laser  range-finder. 

The  nature  of  the  ground  evaluation  remains  the  same,  the 
performances  of  the  small  VAP  in  these  conditions  become: 

Range  measurement: 

Distance:  10  m 

Field  of  view:  50deg. 

(only  one  range-finder) 

Total  mass:  6.5  kg 

Power  consumption:  8  Watts 

(during  5  sec  each  time) 

0.5W  standby 
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5.5-  THE  POWER  SUPPLY 

The  power  supply  of  a  martian  rover  must  deliver  enough 
power  during  the  complete  lifetime  (1  year). 

After  a  quick  survey  on  the  possible  power  generators,  only 
two  remaining  solutions  can  be  envisaged: 

-  Solar  panel. 

-  RTG  (radio-isotop  thermal  generator). 

A  rapid  power  consumption  budget  for  the  complete 
vehicle,  shows  that  the  need  is  very  high: 

-  250  Watts  for  the  big  VAP. 

-  60  Watts  for  the  small  VAP. 

A  quick  estimate  of  the  solar  panel  area  needed  for  such 
level  of  power  forces  to  reject  this  solution: 

The  rover  need  this  power  even  in  the  worst  case  (sun  low 
above  the  horizon,  in  winter  season,  number  of  daylight 
hours,  panel  orientation,  etc....)  which  leads  to  several 
square  meters  of  panels.  In  addition,  this  solution  is  also 
rejected  because  of  the  induced  problems  caused  by  the 
possible  wind,  the  dust,  and  the  necessary  orientation  which 
will  decrease  drastically  the  efficiency. 

The  only  possible  solutionis  therefore  the  RTG.  The  power 
subsystem  will  be  built  around  such  a  generator  associated 
with  batteries  for  the  peak  current.  Most  of  the 
consumption  is  dissipated  when  moving. 

A  good  compromise  must  be  made  taking  into  account  all 
the  parameters  (which  generally  influence  the  power  in 
opposite  ways): 

-  Moving  speed. 

-  Rover  capability  (maximum  slope). 

-  Ground  difficulties. 

-  Battery  size. 

-  Battery  depth  of  discharge  (DoD). 

-  Battery  charge  time. 

-  Peak  current. 

-  Standby  power  consumption. 

-  Rover  activity  (standby,  acquisition,  localization,  vision, 
move,  etc....). 

-  Time  sharing  or  parallel  tasks. 

-  Etc... 

A  specifically  developed  software  was  used  to  find  the 
needed  compromises,  and  to  notice  the  influence  of  several 
parameters  as  the  battery  type. 

As  an  example  of  the  difficulty  of  the  problem  is  given  by 
the  batteries:  which  battery  to  choose  and  what  capacity  ? 
Knowing  that  the  DoD  (depth  of  discharge)  is  not  the  same 
for  all  and  that  the  total  weight  of  the  battery  influences 
the  power  consumption  of  the  electrical  motors  when 
climbing  a  slope. 


Finally,  the  main  performances  of  the  power  sub-system 
are  the  following. 

For  the  large  VAP: 

-  assuming  a  speed  of  0.25  m/sec.: 

RTG  electrical  power:  189  Watts 
batteries  type:  Li-C 
battery  total  capacity:  25  Ah. 

28V  regulated  bus  line 

For  the  small  VAP: 

-  assuming  a  speed  of  0.20  m/sec.: 

RTG  electrical  power:  60  Watts 
batteries  type:  Li-C 
battery  total  capacity:  25  Ah. 

28V  regulated  bus  line 

5.6-  THERMAL  ASPECT 

The  thermal  aspect  of  the  overall  vehicle  is  quite  similar  to 
a  normal  satellite.  A  complete  trade  off  study  has  been 
made  on  several  possible  solutions. 

Two  of  them  seem  to  be  the  best  compromise: 

-  with  thermal  louvers:  they  must  withstand  the  Mars 
environmental  conditions  (dust,  choc)  which  makes  their 
mechanical  design  slightly  more  difficult, 

-  with  phase  changing  materials:  whose  main  problems  are 
mass  and  accommodation. 

6-  CONCLUSION 

This  study  was  performed  recently,  it  shows  that  because 
of  the  great  distance  between  Mars  and  Earth,  a  very  large 
autonomy  is  required  for  a  rover  to  explore  the  surface  of 
the  quite  unknown  red  planet. 

Moreover,  this  requested  autonomy  seems  feasible  today 
thanks  to  the  recent  investments  made  in  that  field,  shown 
by  several  demonstration  models. 

Another  result  indicates  that  a  great  part  of  the  onboard 
power  is  dissipated  by  the  motors  during  the  moves,  but 
today,  the  only  possible  power  generation  envisaged  is 
through  RTG. 

Mars  is  veiy  far  from  Earth,  but  all  the  problems  involved 
with  a  vehicle  roving  on  its  surface  can  be  solved  today. 
Nevertheless,  deeper  studies  are  necessary  now  if  we  want 
to  be  in  time  for  exploring  large  areas  of  the  martianground, 
necessary  to  complement  the  underground  studies  already 
envisaged  by  means  of  fixed  stations. 
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GUIDANCE  AND  CONTROL  FOR  UNMANNED  GROUND  VEHICLES 


P  .J.  Bateman, 

"Allerford",  6  Brackendale  Road, 
Camberley,  Surrey,  GUI 5  2JN.  U.K. 


SUMMARY 

Techniques  for  the  guidance,  control  and  navigation 
of  unmanned  ground  vehicles  are  described  in  terms 
of  the  communication  bandwidth  requirements  for 
driving  and  control  of  a  vehicle  remote  from  the 
human  operator.  Modes  of  operation  are 
conveniently  classified  as  conventional  teleoperation, 
supervisory  control  and  folly  autonomous  control. 
The  fundamental  problem  of  maintaining  a  robust 
non  line  of  sight  communications  link  between  the 
human  controller  and  the  remote  vehicle  is  discussed, 
as  this  provides  the  impetus  for  greater  autonomy  in 
the  control  system  and  the  greatest  scope  for 
innovation.  Whilst  supervisory  control  still  requires 
the  man  to  be  providing  the  primary  navigational 
intelligence,  folly  autonomous  operation  requires  that 
mission  navigation  is  provided  solely  by  on-board 
machine  intelligence.  Methods  directed  at  achieving 
this  performance  are  described  using  various  active 
and  passive  sensing  of  the  terrain  for  route 
navigation  and  obstacle  detection.  Emphasis  is  given 
to  TV  imagery  and  signal  processing  techniques  for 
image  understanding.  Reference  is  made  to  the 
limitations  of  current  microprocessor  technology  and 
suitable  computer  architectures. 

Some  of  the  more  recent  control  techniques  involve 
the  use  of  neural  networks,  frizzy  logic  and  data 
fusion  and  these  are  discussed  in  the  context  of  road 
following  and  cross  country'  navigation.  Examples  of 
autonomous  vehicle  testbeds  operated  at  various 
laboratories  around  the  world  are  given. 

1.  INTRODUCTION 

In  order  to  compete  with  an  on-board  human  driver 
for  anything  other  than  specialist  missions,  such  as 
mine  clearance  or  materials  supply,  the  unmanned 
vehicle  will  need  to  be  capable  of  negotiating  its 
route  to  a  destination  far  from  the  starting  point, 
safely,  and  at  a  speed  comparable  to  that  of  a 
manned  vehicle.  The  mission  may  be  in  a  structured 
environment  (e.g.  on  roads)  when  it  may  encounter 
other  static  or  moving  traffic  or  it  may  be  in  a  totally 
unstructured  environment  (across  natural  terrain) 
when  the  primary  task  will  be  to  locate,  recognise 
and  avoid  obstacles  in  its  path. 

Whilst  artificial  intelligence  is  currently  far  from 
solving  all  these  problems  in  an  integrated  and  cost 
effective  fashion,  at  the  present  time  it  is  appropriate 
to  review  the  progress  that  has  been  made  in  the  field 
and  to  attempt  to  predict  the  advances  that  can  be 
expected  in  the  future. 

An  unmanned  ground  vehicle  (UGV)  will  be  operated 
in  one  of  a  number  of  control  modes  and  it  will  be 
convenient,  for  the  purposes  of  this  paper,  to  define 
these  first  as  they  serve  as  a  useful  framework 


around  which  to  describe  the  guidance  and  control 
technology. 

In  the  basic  remote  control  mode  the  operator  sends 
to  the  vehicle,  by  radio  or  land  line,  steer  and  speed 
commands.  He  then  views  the  response  of  the 
vehicle  directly  in  the  field  of  view'  of  his  eye,  or  a 
suitable  optical  sight,  and  continues  to  send 
command  signals  to  guide  the  vehicle  to  its 
destination.  Clearly  this  will  generally  only  be 
practicable  if  the  operator  has  a  clear  unimpeded 
view  of  both  the  vehicle  and  its  intended  destination 
at  all  times.  This  will  be  a  major  restriction 
operationally.  It  does,  however,  require  only  a  one 
way  communication  link,  the  data  requirements  for 
which  are  common  to  the  more  general  case  of 
teleoperation.  Here  the  vehicle  is  fitted  with  a 
television  camera  and  this  overcomes  the  need  for  the 
human  operator  to  have  line  of  sight  to  the  vehicle. 
In  this  case,  the  video  signal  is  transmitted  from  the 
vehicle  and  received  by  the  command  station  where 
the  operator  views,  on  his  monitor,  the  visual  scene 
ahead  of  the  vehicle.  The  operator  then  responds  by 
sending  the  speed,  steer  and  other  commands  to  the 
vehicle  on  what  is  now  a  duplex  communication  link. 
These  signals  will  be  of  very'  low  data  rate  compared 
with  the  bandwidth  required  for  the  video  signal. 
The  bandwidth  required  for  a  standard  TV  picture 
will  be  of  the  order  of  5MHz  or  so  and  it  is  unlikely 
that  this  will  be  available  in  the  radio  spectrum  for 
dedicated  operation,  unless  special  provision  for  line 
of  sight  airborne  or  satellite  relays  is  made.  For 
ground  to  ground  transmission,  the  high  carrier 
frequency  required  to  bear  the  video  signal  is  not  in 
general  compatible  with  non-line  of  sight  operation, 
ft  is  this  latter  requirement  which  pushes  the 
communication  carrier  frequency  into  the  VHP  part 
of  the  spectrum  for  radio  control.  If  fibre  optic 
ground  links  are  acceptable  from  an  operational  point 
of  view,  then  the  bandwidth  restriction  no  longer 
applies  as  multi  channel  video  capacity  is  easily 
available. 

If  VHF  carrier  frequencies  are  to  be  used  for 
teleoperation,  then  the  video  signal  from  the  TV 
camera  on  the  vehicle  has  to  be  compressed  to  a  data 
rate  compatible  with  this  frequency  band  for 
transmission  back  to  the  operator.  This  compression 
has  to  be  achieved  by  carrying  put  a  degree  of  image 
processing  on  board  the  vehicle.  Typically  we 
consider  a  data  link  of  about  1 6kbits/sec  capacity  or 
less  as  the  link  data  rate  for  this  lower  bandwidth 
teleoperation.  Compared  with  the  100  Mbit/sec  link 
that  is  needed  for  good  quality  televisipn  imaging  in 
the  teleoperation  mode,  this  requires  a  data 
compression  of  6000  to  one  and  will  inevitably  result 
in  considerably  degraded  imagery  .  It  can  be  seen 
therefore  that  teleoperation  requires  a  man  in  the  loop 
at  all  times,  with  the  corresponding  stress  associated 
with  the  task,  and  it  requires  a  communication  link  of 
high  integrity  to  be  maintained. 
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The  supervisory  mode  of  control  seeks  to  overcome 
these  disadvantages.  It  requires  the  operator  to  have 
relatively  frequent  but  intermittent  communication 
with  the  vehicle.  Local  destination  or  waypoints  are 
specified  by  the  human  operator,  and  an  on-board 
autopilot  navigates  the  vehicle  automatically  over  the 
terrain  to  these  destination  points  in  turn  Obstacle 
detection  and  avoidance  techniques  are  designed  into 
the  vehicle  system  so  that  it  can  navigate  safely.  The 
human  operator  becomes  a  supervisor,  designating 
the  route  and  overseeing  the  performance  of  the 
vehicle.  The  reduced  operator  workload  will  allow 
multiple  vehicles  to  be  controlled  by  a  single 
operator. 

This  is  one  step  towards  fully  autonomous  control 
which  will  allow  the  communication  requirements  to 
be  reduced  to  their  lowest  level.  Here  the  vehicle  is 
tasked  with  a  mission  plan  in  advance  and  it  sets  out 
to  implement  the  plan,  using  its  own  artificial 
intelligence  to  decide  on  alternative  courses  of  action, 
and  selecting  in  turn  those  most  suitable  for  fulfilling 
the  overall  mission  successfully. 

The  human  operator,  however,  still  needs  to  maintain 
a  duplex  communication  link  with  the  vehicle  for  the 
purpose  of  monitoring  its  performance,  re-tasking, 
and  sending  corrective  signals  to  the  vehicle  if  called 
for  in  extremis.  This  autonomous  mode  of  control 
requires  a  considerable  degree  of  on-board 
intelligence  with  the  human  operator  being  involved 
only  in  an  emergency.  Perhaps  the  most  important 
aspect  here  is  that  the  on-board  intelligence  should, 
at  each  decision  point,  generate  a  highly  robust  and 
reliable  solution  so  that  the  overall  mission,  which 
comprises  very  many  such  decisions  will  be 
successful.  It  is  this  search  for  robustness  in 
decision  making  which  drives  much  of  the  long  term 
research  in  this  area. 

Fig.  1  summarises  the  different  modes  of  control  in 
terms  of  a  speculative  development  timescale  and  a 
broad  indication  of  the  communication  data  rates 
envisaged. 
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Fig  1  Development  of  robotic  technology 


Having  defined  the  different  modes  of  control  we 

Srocccd  to  address  the  technical  aspects  in  more 
reviewing  the  problems  of  maintaining  the 
communication  link,  the  minimum  data  rate 
requirements,  and  the  human  factor  issues  involved. 
Considerable  attention  is  given  to  autonomous 
control  as  this  presents  exciting  technical  challenges 
in  tire  fields  of  computer  vision,  vehicle  control  and 
mission  management.  The  paper  ends  with 


descriptions  of  some  of  the  laboratory  test  beds 
which  have  been  designed  to  demonstrate'  state  of  the 
art  performance. 

2.  TELEOPERATION 

The  ability  to  drive  safely  using  teleoperation  will 
depend  on  a  number  of  factors:  the  luminance  of  the 
scene,  the  effective  resolution  and  field  of  view  of  the 
optoelectronic  system  and  the  effective  visual  acuity 
of  the  operator.  It  is  interesting  to  compare  the 
potential  overall  visual  acuity  of  a  driver  of  a 
manned  vehicle  with  that  of  a  human  operator 
driving  a  remote  vehicle  by  teleoperation  (Fig. 2).  If 
we  assume  that  the  remote  driver  is  viewing  a  30cm 
high  monitor  screen  at  a  distance  of  50cm  (i.e.  a 
subtended  angle  of  34°)  and  that  the  communication 
bandwidth  restricts  his  display  to  625  lines  per 
picture  height,  then  the  angular  subtense  of  a  single 
line  is  about  i  milliradian.  The  eye  resolution  is 
strongly  dependent  on  contrast  and  the  population 
sample,  but  a  value  of  0.5  milliradian  is  typical  for 
good  contrast  and  luminance  conditions,  so  this  is  not 
a  limitation.  If  we  compare,  therefore,  an  on  board 
driver  with  one  telcoperating  a  remote  vehicle  with 
unityr  magnification,  i.e.  the  scene  subtense  is  the 
same  for  both  drivers,  then  it  does  not  appear  from 
geometrical  conditions  alone  that  the  teleoperation  is 
suffering  much  disadvantage.  However,  the 
teleoperator  has  the  optoelectronic  system  inserted  in 
his  vision  train  and  such  a  system  will  in  fact  degrade 
his  image  by  the  modulation  transfer  function  of  the 
optical  system,  the  noise  and  limiting  responsivity  of 
the  video  camera  and  electronics,  and  the  interlacing 
of  the  display.  Care  is  necessary',  therefore,  in  the 
design  of  the'  system  to  minimise  die  degradation  in 
quality'  of  the  displayed  image.  Alternatively,  the 
image  quality  can  be  restored  by  narrowing  the  field 
of  view'  of  the  TV  camera  but  this  can  be  self- 
defeating  as,  for  safe  driving,  a  good  peripheral  field 
of  view  is  essential. 

3.  THE  MARDI  SYSTEM 

The  MARDI  demonstrator  system  is  a  good 
example  of  a  teieoperated  vehicle  and  illustrates 
these  general  principles  (Ref.  1).  The  remote  vehicle 
was  tiased  on  a  5  ton  tracked  platform,  one  of  the 
family  of  British  armoured  reconnaissance  vehicles. 
In  order  to  give  a  wide  horizontal  field  of  view  for 
the  driving  function,  three  cameras  were  mounted  in 
a  compartment  on  the  forward  edge  of  the  roof  of  the 
vehicle  each  being  angled  with  contiguous  fields  of 
view  to  give  an  overall  horizontal  field  of  100 
degrees.  The  unit  could  be  controlled  in  pitch 
relative  to  the  vehicle  to  ease  the  problem  of  driving 
in  undulating  country.  It  was  found  that  for  driving 
it  was  the  central  camera  display  that  was  used  most 
of  the  time,  but  the  peripheral  cameras  were  valuable 
for  turning  and  maneouvring.  As  the  radio 
communication  svstem  was  limited  to  one  foil 
bandwidth  TV  channel,  a  multiplexing  system  was 
designed  to  give  a  reduced  resolution  image  on  the 
central  display  and  a  still  further  degraded  image  on 
the  two  outer  displays  to  avoid  exceeding  the  channel 
capacity'. 


From  a  human  factors  viewpoint,  the  object  was  to 
give  the  operator  a  real  feeling  of  telepresence  by 
providing  as  many  cues  as  possible  to  simulate  the 
driving  function  of  a  manned  vehicle.  A  steering 
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yoke,  accelerator  and  brake  pedals  were  identical  to 
those  used  in  the  Warrior  armoured  fighting  vehicle. 
A  microphone  in  the  driver's  compartment  of  the 
vehicle  was  used  to  give  feedback  of  engine  and 
gearbox  noise  to  the  remote  driver.  This  was  useful 
in  providing  confirmation  to  the  driver  that  the 
vehicle  was  responding  satisfactorily  to  command 
signals. 

An  inertial  navigation  system  was  fitted  to  provide 
both  vehicle  position  and  an  indication  of  heading, 
pitch,  and  roll  angles.  The  latter  were  presented  as  a 
symbolic  overlay  on  the  central  display,  together  with 
vehicle  speed  and  various  warning  indications  at  the 
edge  of  the  display.  The  vehicle  position  was 
displayed  on  a  digital  map  mounted  above  the  central 
Tv  driving  momtor.  The  map,  scaled  at  1:25,000, 
was  scrolled  to  keep  the  vehicle  central  on  the  map. 
Information  on  the  instantaneous  heading  and  the 
planned  route  was  also  superimposed. 

The  value  of  good  map  information  was  very  evident 
during  the  navigational  trials  This  allows  potential 
landmarks  and  terrain  features  en  route  to  be 
prepared  beforehand  during  the  mission  planning 
phase.  Visibility  of  these  landmarks  ana  terrain 
features  can  be  anticipated  by  inspection  of  the  local 
topography  and,  during  the  actual  mission  itself,  this 
information  can  be  used  by  the  crew  (driver  and 
commander)  to  confirm  the  correctness  of  the  route. 
Often  the  camera  viewpoint  can  be  lower  than  typical 
manned  driving  positions,  and  waypoint  data  to  help 
guide  navigation  may  be  obscured,  increasing  the 
chance  of  the  vehicle  becoming  lost  in  cross  country' 
navigation  exercises. 

In  the  MARDI  system,  apart  from  a  safety  operator 
(necessary  for  experimental  trials)  the  crew  included 
a  driver  whose  responsibility  was  to  control  the 
vehicle,  and  operate  the  payload  (e  g.  surveillance 
system)  when  the  vehicle  was  stationary'.  The  second 
of  the  two  primary  crewmen  was  the  Commander, 
whose  task  was  to  plan  and  monitor  the  progress  of 
the  mission,  and  advise  the  driver  on  any  particular 
features,  obstacles,  etc.  which  he  could  identify 
through  the  use  of  a  panoramic  surveillance  head 
during  the  mission.  This  surveillance  sight  was 
mounted  on  an  elevated  mast  and,  with  extra 
magnification  available,  this  allowed  a  more  detailed 
appreciation  of  the  surroundings  to  be  obtained  than 
was  the  case  for  the  driver.  However,  the  system 
was  basically  designed  for  one  man  operation  only, 
the  driver  being  able  to  operate  all  the  essential 
controls  himself. 

The  control  system  used  for  MARDI  is  shown  in  Fig. 

3.  The  Command  Centre  comprises  a  box  body 
mounted  on  a  4  ton  Bedford  truck.  The  Driver, 
Commander  and  Safety  Operator  are  seated  at 
separate  consoles  each"  with  their  own  monitor 
displays  and  controls.  Both  tire  remote  vehicle 
control  and  the  command  centre  control  are  based  on 
68020  microcompressors  with  VME  bus 
arrangements,  and  they  are  connected  by  the 
communications  link  which  is  a  selectable  fibre  optic 
or  radio  system.  In  addition,  there  is  a  separate 
safety  system  called  a  Dead  Man’s  Handle  (DMH) 
which  operates  on  a  separate  radio  channel  and  is 
capable  of  disenabling  the  vehicle  if  there  is  an 
emergency. 


The  remote  vehicle  controller  operates  either  to 
control  the  platform  or  the  payload  under  the 
command  of  the  driver.  Modifications  were  made  to 
the  vehicle  to  make  it  suitable  for  computer  control. 
These  included  an  automatic  gearbox  control  unit,  a 
hydraulic  braking  and  steering  system,  a  servo 
throttle  control  system  and  an  automatic  fire 
suppresion  system.  In  addition,  there  is  computer 
control  of  ignition,  starter,  and  hydraulic 
accumulator  charging  system,  and  feedback  of 
warnings  such  as  low'  oil  pressure  are  provided.  The 
payload  consists  of  three  separate  facilities:  a 
surveillance  head,  weapon  system  and  a  smoke 
generator.  The  surveillance  head  comprises  a 
daylight  TV  and  thennal  imaging  sight  ana  a  laser 
rangefinder.  The  weapon  system  comprises  a  light 
anti  armour  weapon  training  simulator,  a  chain  gun 
and  a  laser  projector  derived  from  a  direct  fire 
weapon  effects  simulator. 

4.  MARDI  COMMUNICATIONS 

The  primary  communications  link  between  the 
command  Centre  and  the  Remote  Vehicle  is  a  single 
mode  fibre  optic  system  (Fig.  4).  The  system  is 
designed  to  work  over  a  fibre  optic  distance  of  up  to 
36km  but.  in  practice,  the  length  of  cable  which  can 
be  accommodated  on  the  remote  vehicle  is  the 
limiting  factor.  This  is  14km  for  single  use 
disposable  cable  drums,  or  4km  for  cables  which  can 
be  rewound  after  use  for  repeated  operation.  A  4- 
wheel  drive  Honda  motor  cycle  was  adapted  for 
recover}'  of  cable  previously  dispensed  by  the  remote 
vehicle.  Both  disposable  and  recoverable  cables  are 
simply  pulled  from  the  (non-rotating)  drum  by 
friction  with  the  ground  as  the  remote  vehicle  moves 
forward,  and  this  has  been  found  satisfactory  up  to 
the  maximum  speed  of  the  vehicle,  namely  about 
50  kph. 

The  fibre  optic  system  can  transmit  five  full  qualify' 
PAL  video  images  from  the  remote  vehicle  to  the 
Command  Centre  together  with  an  audio  channel. 
Also  data  can  be  sent  in  both  directions  at 
19.2kbits/s. 

The  second  communications  medium  is  a  microwave 
link  operating  at  1.4GHz  (down  link)  and  2.4GHz 
(up  link).  A  steerable  dish  antenna  is  mounted  on  the 
Command  Centre,  and  the  Remote  Vehicle  is  fitted 
with  an  omni-directional  whip  antenna.  The 
microwave  link  transmits  digital  command  data  at 
19.2kbits/sec  from  the  Command  Centre,  and 
digitised  video,  audio  and  data  from  the  Remote 
Vehicle,  the  video  being  limited  to  only  one  full 
qualify  PAL  image. 

5.  SAFETY  SYSTEMS 

Dealing  with  potentially  dangerous  vehicles  from  the 
viewpoint  of  the  damage  which  they  may  inflict  if 
they  get  out  of  control,  focuses  attention  on  a  very 
important  feature  of  design.  The  overall  philosophy 
of  safety  and  the  impact  this  has  on  the  design  has  to 
be  considered  at  the  earliest  stages  in  the  project. 
The  scope  of  such  a  study  has  to  take  into  account 
the  collision  of  a  rogue  vehicle  and  any  safety 
aspects  associated  with  the  piayload  such  as 
operation  of  on-board  weapons.  The  safety  system 
design  may  well  differ  between  an  experimental 
demonstrator  system,  such  as  MARDI,  and  a 
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potential  in-service  vehicle.  In  the  case  of  the 
Former,  safety  is  of  paramount  concern  in  the  trials 
and  evaluation  of  new  and  untried  subsystems.  For 
inservice  vehicles  their  particular  role  may  allow 
certain  safety  features  to  be  dispensed  with. 

At  the  highest  level,  for  development  work  on  trials 
ranges,  a  limited  fuel  supply  system  has  been 
installed  which  ensures  that  the  vehicle  will,  under  all 
conditions  of  operation,  exhaust  its  fuel  supply 
before  running  outside  the  trials  safety  area. 

For  MARDI  a  dedicated  Trials  Safety  Officer  was 
accommodated  in  the  Command  Centre  with  his  own 
operator's  console.  This  is  fitted  with  a  video 
monitor  for  camera  images,  a  map  display  monitor, 
error  displays  and  system  warnings,  and  an 
emergency  shut  down  button.  The  Commander  and 
Driver  both  have  emergency  stop  buttons  to  stop  the 
vehicle  or  halt  weapon  firing  by  sending  disenabling 
signals  over  the  up  command  link.  Emergency  stop 
action  on  the  part  of  the  crewmen  will  usually  be  in 
response  to  indications  on  the  monitor  displays  which 
in  themselves  rely  on  the  integrity  "of  the 
communications  link.  In  the  event  of  a  loss  of 
communications  occurring  or  an  unsafe  system  state 
(e.g.  low  hydraulic  pressure  which  supplies  the 
braking  system),  an  automatic  computer  controlled 
shutdown  is  initiated  which  halts  the  vehicle  or 
inhibits  its  weapon  system.  There  is  still  the 
possibility  of  a  computer  hardware  or  softw  are  fault 
occurring  and,  to  cover  this  situation,  a  computer 
"watchdog"  on  the  Remote  Vehicle  will  apply  the 
main  brakes  if  faults  are  detected  in  this  or  the 
communications  system  (Ref.  2). 

Up  till  now  all  the  safety  commands  have  been 
transmitted  via  the  main  communications  link 
between  the  Command  Centre  and  the  Remote 
Vehicle.  This  is  clearly  a  potential  source  of 
weakness  so,  again  for  the  purposes  of  development 
trials,  a  Dead  Man's  Handle  has  been  introduced.  It 
is  based  on  an  independent  fail  safe  radio  link 
operated  from  a  manned  Land  Rover  vehicle.  In  all 
circumstances,  this  vehicle  stays  within  line  of  sight 
contact  of  the  Remote  Vehicle,  and  the  Dead  Man's 
Handle  operator  maintains  the  radio  link  to  enable 
the  Remote  Vehicle  to  move  under  control.  The 
range  of  safety  features  may  appear  excessive,  but 
when  little  experience  exists  in  the  operation  of  large 
UGV's  caution  is  wise.  As  confidence  is  gained 
simplifications  can  be  contemplated.  The  Dead 
Man's  Handle  could  be  operated  from  the  Command 
Centre  itself  instead  of  from  a  separate  vehicle  and 
the  limited  fuel  system  could  be  dispensed  with.  For 
the  operation  of  weapons  systems  a  separate  set  of 
control  procedures  is  necessary.  Currently  a 
keyswiten  on  the  Remote  Vehicle"  has  to  be  enabled 
to  allow  arming  of  the  system,  and  firing  the  weapon 
system  can  only  take  place  after  a  series  of  specific 
commands  is  received  from  the  Command  Centre  and 
the  DMH  is  enabled.  Much  still  remains  to  be  done 
in  this  area  to  optimise  the  safety  system  for  weapons 
operation,  particularly  in  conjunction  with  movement 
of  the  remote  vehicle  where  a  combination  of  vehicle 
motion  and  the  carriage  of  live  ammunition  on  board 
poses  an  additional  hazard. 


6.  COMMUNICATIONS  PROTOCOLS 

There  are  broadly  two  types  of  data  which  have  to  be 
exchanged  between  the  Remote  Vehicle  and  the 
Command  Centre.  These  are  (1)  the  relatively  high 
data  rate  imaging  data  on  the  down  link  with  data 
rates  in  the  range  10kHz  to  5MHz  and  (2)  the  low 
data  rate  control  and  feedback  data  on  both  the  up 
and  down  links.  Radio  will  provide  the  most  flexible 
but  moreover  the  most  vulnerable  means  of 
communication.  Channel  capacity  will  always  be  at 
a  premium  and  a  substantial  amount  of  this  capacity 
will  be  taken  up  with  error  protection  to  counter 
interference  and  jamming.  The  remainder  will  be 
dedicated  to  the  protocol  necessary  to  multiplex  and 
address  the  real  signal  data.  For  any  unmanned 
ground  vehicle  system,  therefore,  there  "is  an  urgent 
need  to  minimise  the  bit  rate  associated  with  both  the 
control  data  and  the  protocol  overhead.  The  imaging 
data  tends  to  be  a  special  case  defined  by  its  video 
nature,  whilst  the  control  data  may  comprise  many 
channels  of  continuous  repetitive  data  and  one  shot 
function  operations.  We  shall  now'  discuss  what  is 
involved  in  minimising  the  protocol  overhead  and  the 
quantitative  data  itself. 

Some  form  of  protocol  is  necessaiy  to  arrange  the 
essentially  multi-input  data  into  a  form  suitable  for 
serial  transmission  and  subsequent  decoding  by 
computer.  The  pressures  for  longer  term 
interoperability  and  ease  of  adaptation  lend  support 
to  the  need  for  standardisation  at  an  early  stage.  The 
International  Organisation  for  Standardisation  (ISO) 
is  a  voluntary  organisation  formed  to  promulgate 
standards.  In  1978  a  reference  model  OSI-RM  was 
introduced  as  a  model  of  a  computer  communications 
architecture.  Its  aim  was  to  promote  compatible 
communications  among  a  wide  variety  of  digital 
systems  and  to  provide  a  framework  for  developing 
standards.  OSl-RM  is  not  a  protocol  standard.  It 
specifies  seven  distinct  layers  (Fig.  5)  which  define 
the  functions  involved  in  communicating,  and 
protocols  have  to  be  developed  at  each  layer.  Each 
layer  must  communicate  with  the  layer  above  and 
below  it.  For  communicating  between  different 
systems,  it  is  imperative  that  each  system  has 
identical  layers  and  uses  the  same  protocols. 

The  bottom  three  layers  of  OSI-RM  are 
communication  layers,  concerned  with  the 
transmission  of  bits  and  bytes.  These  layers  are 
hardware  dominated.  The  top  three  layers  are  the 
information  processing  layers,  adding  intelligence  to 
the  bits  and  bytes  being  transmittecL  These  layers 
are  software  dominated.  The  fourth,  or  middle,  layer 
bridges  the  gap  between  these  two  sets  of  layers. 

In  detail,  the  Medium  refers  to  the  type  of  channel 
over  which  the  signal  is  transmitted,  e  g.  air,  fibre 
optic  cable.  The  Physical  layer  covers  the  physical 
interface  between  the  devices  used  to  send,  and 
receive  bits  over  the  medium,  e.g.  modulating  and 
demodulating  the  signals.  The  Data  Link  layer 
attempts  to  make  the  physical  layer  reliable,  e.g.  by 
bit  error  detection  ana/or  correction,  synchronisation 
between  sender  and  receiver,  and  vehicle  addressing 
for  multi-vehicle  operation.  To  perform  these 
functions  the  bits  are  arranged  into  frames.  These 
frames  add  overhead  bits  which  are  used  to  perform 
the  functions  of  this  layer.  The  Network  layer  routes 
data  through  a  network  of  computers/terminals. 
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Fig  5  OSI-RM 


Initially  during  development,  UGV's  will  require 
simple  point  to  point  communications  and  this  layer 
will  not  be  needed.  However,  when  UGV's  are 
fielded  in  service,  networking  may  well  be  required. 

The  Transport  layer  provides  the  bridge  between  the 
three  lower  communication  layers  and  the  three 
higher  information  processing  layers.  It  does  for  an 
end  to  end  link  what  the  data  “link  layer  does  for 
point-to-point  links.  The  services  provided  are 
sequencing,  expedited  delivery  and  survival  of  the 
connection.  The  key  feature  is  that  expedited 
deliven-  is  implemented  using  a  three  level  priority 
label  tor  the  transmission  of  information  packets. 
The  label  defines  whether  an  acknowledgement  is 
needed  or  not.  The  Session  layer  provides  the 
mechanism  for  controlling  the  dialogue  between 
applications.  It  provides  one  of  three  types  of 
dialogue:  full-duplex,  half  duplex  or  simplex.  It 
establishes  and  terminates  connections  and  it  allows 
abrupt  or  graceful  disconnections.  In  other  words,  it 
controls  whether  a  message  can  or  cannot  be 
disrupted  in  the  midst  of  a  transmission,  and  also  the 
action  to  be  taken  if  the  message  is  interrupted.  The 
Presentation  layer  ensures  that  the  information  is 
delivered  in  a  “form  that  the  receiving  system  can 
understand.  This  is  primarily  for  providing  security 
through  encryption,  message  compression,  and 
svntax  conversion,  and  is  more  appropriate  for  future 
fielded  systems. 

The  Application  layer  is  to  the  user  the  layer  which 
appears  to  be  doing  the  real  work.  It  describes  the 
bit  and  byte  representations  of  the  control  commands 
and  other  status  information.  A  common  message 
format  has  been  developed  by  the  US  Army  Tank 


Automative  Command  (Ref.3)  specifically  for 
robotic  vehicles.  This  Robotic  Vehicle  Message 
Format  (RVMF)  is  in  a  block  format  in  which  each 
control  parameter  is  preceded  by  a  function 
identifier.  It  has  the  flexibility  that  different 
parameters  can  be  sent  at  different  time  intervals. 
The  message  length  is  variable.  When  little  or  no 
activity'  is  required,  the  message  length  will  approach 
zero  bytes.  A  UGV  performing  a  variety  of 

functions  in  a  complex  environment  will  have  a  long 
message  length.  It  also  has  the  advantage  that  it  can 
order  commands  by  priority.  The  penalty,  however, 
is  that  it  does  require  an  increased  amount  of 
processing.  This  can  be  appreciated  by  examining 
the  overall  Message  Format  whicli  is  shown  in  Fig.  6 
and  gives  the  bit  numbers  associated  with  each 
message  and  submessage.  The  transaction  represents 
the  actual  information  that  is  to  be  transmitted  and 
includes  (1)  a  sequence  number  to  identify  the 
acknowledgment  returned  against  the  original 

message,  (2)  a  transaction  type  which  describes  the 
information  type  and  defines  now  the  recipient  should 
treat  the  information,  (3)  the  attribute  which 
identifies  the  control  command  or  the  status 
information  that  is  being  conveyed  and  (4)  the 

parameter  itself  expressed  in  terms  of  the  numeric 
value  required,  the  selection  of  a  particular  option, 
e.g.  gear  change,  or  a  prescribed  action  as  a  function 
of  time.  The  transaction  type  comprises  the 

disposition  which  defines  the  status/purpose  of  the 
command,  e.g.  initiated  and  executed,  and  the 
category'  which  defines  the  type  of  command,  e.g. 
control  with  or  without  a  requirement  for 
acknowledgment. 

The  total  message  length  is  limited  to  128  bytes  in 
order  to  minimise  the  latency,  that  is,  the  delay 
between  repeated  updates  of  a”  particular  command.. 
A  message  size  of  128  bytes  limits  the  latency  time 
to  about  53ms  at  a  channel  bit  rate  of  19.2kbits/sec. 
But  as  we  shall  see  later  we  shall  be  restricted  to 
much  lower  bit  rates  for  control  data  for  field  worthy 
sy  stems,  and  this  will  correspondingly  increase  the 
latency  time.  We  conclude,  therefore,  that  the 
RVMF  is  very  flexible  and  suitable  for  development 
purposes  but  may  attract  an  unacceptable  overhead 
For  particular  systems  applications.  Current  studies 
are  in  progress  to  improve  the  efficiency  of  the 
RVMF  and  thereby  reduce  the  overhead. 

The  MARD1  system  has  adopted  the  R  VMF  protocol 
and  trials  have  been  carried  out  to  explore  the 
minimum  data  rate  that  will  allow  such  a  vehicle  to 
be  controlled  satisfactorily. 

In  a  real  time  control  system,  there  are  three  major 
parameters  associated  “with  repetitive  commands, 
namely  the  update  rate,  the  delay  and  the  resolution. 
The  update  rate  clearly  determines  the  latency  of  the 
command  information!  On  average  the  system  will 
not  receive  the  information  to  act  on  until  half  the 
repetition  period  has  elapsed.  Delays  will  be 
associated  with  computing  time  and  the  response 
time  of  the  actuation  system  which  is  Being 
energised.  The  resolution  defines  the  precision  of  the 
demand  which  is  placed  on  the  actuator  system.  In 
controlling  a  remote  vehicle  three  parameters  will  be 
involved,  namely  steering,  throttle,  and  brakes. 
With  MARDI  it  was  found  that  satisfactory  control 
could  be  achieved  for  these  parameters  with  a 
resolution  of  4,  3,  and  2  bits  respectively,  showing  a 
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considerable  reduction  on  the  8  bits  previously  used 
for  each  parameter.  Adding  an  allowance  for  other 
non-continuous  control  signals  such  as  gear 
selection,  a  total  driving  message  of  16  data  bits  may 
be  sufficient.  In  order  to  decode  this  message  a 
protocol  would  need  to  be  added  to  this  data. 
Assuming  that  a  protocol  specially  tailored  to  the 
application  is  available,  a  further  16  bits  should  be 
added  for  this  purpose,  giving  a  32  bit  message^ 
Error  correction  to  enable  the  data  to  be  transmitted 
over  a  noisy  link  might  add  a  further  100%  giving  a 
total  message  length  of  64  bits.  In  order  to  keep  the 
message  delay  below  100  ms  the  message  must  be 
transmitted  in  100  ms  giving  a  transmission  rate  of 
10Hz.  Thus  we  arrive  at  a  command  link  operating 
at  640  bits/sec  capable  of  giving  an  adequate  driving 
performance.  This  is  a  considerable  reduction  on  the 
19.2  kbits/sec  currently  being  used. 

7.  COMMUNICATIONS  MEDIA 

The  discussion  on  teleoperation  of  UGV's  has 
highlighted  the  problems  of  communications  as  a 
major  factor  in  determining  the  range  of  applicability 
of  such  a  vehicle  concept.  We  have  seen  that,  apart 
from  communications  issues,  conventional 
teleoperation  raises  no  potentially  insuperable 
technical  problems.  Nevertheless,  the  difference  in 
data  capacity  requirements  between  the  up  and  down 
links  focuses  the  problem  on  the  down  link  where  full 
quality  video  imagery  will  be  demanding  bandwidths 
of  the  order  of  5  MHz.  This  presents  no  difficulty 
for  fibre  optic  systems  and  we  shall  consider  this  in 
more  detail  later  in  this  section.  But  the  real 
attraction  of  the  UGV  concept  is  for  untethered 
operation,  and  the  opportunity  for  vehicles  to  range 
freely  at  considerable  distances  (measured  in 
kilometres)  from  the  command  centre.  It  is 
important,  therefore,  to  review  the  question  of  radio 
communication  for  UGV  application. 

7.1  Radio  Communications 

Apart  from  data  capacity,  there  is  a  range  of  other 
factors  which  has  to  be  considered:  (1)  Path  loss 
absorption  effects,  (2)  Multipath  effects  which  lead 
to  potential  interference  and  fading,  (3)  Physical 
dimensions  and  positioning  of  antenna  systems,  (4) 
Moving  vehicle  effects  (5)  Channel  spacing  and 
allocation  and  (6)  Detection  and  countermeasures  by 
a  potential  enemy.  Many  of  these  are  conflicting  and 
obtaining  an  optimum  solution  will  be  a  complex  and 
difficult  exercise.  The  radio  frequency  spectrum  is 
covered  by  a  range  of  frequency  bands  and  these  will 
now  be  considered  in  turn. 

Very  Low  Frequency  (VLF)  For  frequencies  below 
30MHz  covering  the  LF  and  HF  bands,  the 
advantages  of  non  line  of  sight  transmission  are 
outweighed  by  the  non-optimum  aerial  configuration 
that  will  be  feasible.  Reception  efficiency  will  be 
optimum  when  the  aerial  is  a  quarter  wave  length 
long  and,  if  we  assume  that  a  maximum  aerial  length 
for  a  UGV  is  2.5m,  then  optimum  reception  will 
occur  at  30MHz  but  will  decrease  progressively  at 
frequencies  below  this  value.  Transmissions  can  be 
detected  at  great  distances,  even  with  low-powered 
transmitters,  given  the  proper  ionospheric  conditions, 
and  a  consequence  of  this  is  that  video  broadcasts 
could  cause  interference  in  locations  far  removed 
from  their  source.  Given  that  bandwidth 


requirements  will  occupy  a  large  proportion  of  the 
total  HF  band,  it  is  inconceivable  that  this  band 
could  be  used  other  than  for  intermittent  low  data 
rate  communication. 

Very  High  Frequency  (VHF)  The  VHF  portion  of 
the  electromagnetic  spectrum  extends  from  30MHz 
to  300MHz.  Its  principal  advantage  over  other 
spectral  regions  is  that  its  range  of  frequencies  is 
great  enough  -  10  times  the  bandwidth  of  the  HF 
band  -  and  at  the  same  time,  the  lower  end  of  the 
VHF  spectrum  can  still  provide  non  line  of  sight  RF. 
It  is,  or  course,  the  primary  waveband  for  short  range 
battlefield  communications.  In  the  UK,  the  current 
inservice  combat  net  Clansman  equipment  will  be 
replaced  by  the  Bowman  system  by  the  end  of  the 
century,  but  the  fundamental  wave  band  does  not 
allow  for  more  than  2000  channels  each  with  a 
bandwidth  of  25kFIz.  Already  there  is  great  pressure 
on  channel  allocation  for  established  users,  and  the 
introduction  of  a  new  requirement  for  UGV’s  means 
that  dedicated  allocation  of  channels  for  this  purpose 
will  only  be  resolved  on  the  basis  of  priorities. 
There  is  a  move  towards  digital  data  transfer  for 
battlefield  combat  net  radios  and  a  data  channel  will 
have  typically  a  capacity  of  16kbits/sec.  This  sets  a 
useful  benchmark  for  the  data  compression  of  video 
for  teleoperation  applications.  However,  this 
requires  compression  ratios  of  3000:1  or  so  and  this 
presents  a  formidable  task  for  robust  digital 
processing  techniques.  It  involves  motion  prediction 
and  interframe  processing  which  may  well  be 
corrupted  by  wmd  and  cloud  motion  in  this 
waveband,  since  above  30MHz  "tropospheric 

Sagation  takes  over  as  the  dominant  transmission 
3,  propagation  being  by  space  waves  within  the 
lower  part  of  the  atmosphere  (Ref.  4). 

Two  other  effects  will  tend  to  degrade  the 
performance  in  this  waveband.  The  well  known 
"track  static"  effect,  arising  from  the  motion  of  a 
tracked  vehicle  over  terrain,  tends  to  reduce  the 
operating  range  below  that  obtainable  w  ith  stationary 
vehicles.  The  second  phenomenon  is  that  due  to 
multipath  effects,  whereby  reflections  from  nearby 
objects,  hillsides  etc.,  will  lead  inevitably  to  additions 
to  and  subtractions  from  the  direct  path  signal.  As 
the  UGV  moves  therefore,  peaks  and  troughs  in  the 
received  signal  will  appear,  as  well  as  a  gradual 
reduction  due  to  the  direct  path  loss.  These  can  be 
expected  to  occur  at  quarter  wavelength  distances. 
For  low  data  rate  control  and  status  signals  these 
effects  can  be  suppressed  as  the  distances  will 
generally  be  short,  e.g.  lm  or  less,  but  as  we  have 
seen  they  may  be  serious  for  video  imaging. 
Polarisation  sensitive  antenna  and  multiple  antennae 
may  help  to  ameliorate  these  effects. 


Ultra  High  Frequency  (UHF)  The  UHF  portion  of 
the  RF  spectrum  extends  from  300MHz  to  3000MHz 
offering  10  times  the  bandwidth  of  the  VHF  band. 
At  frequencies  above  300MHz  the  non  line  of  sight 
advantage  of  the  lower  frequency  ranges  tends  to  be 
lost,  although  at  the  lower  end  of  the  UHF  band  there 
is  a  good  compromise  between  non-directionality  and 
foliage  penetration  on  the  one  hand,  and  capacity  for 
video  transmission  on  the  other. 


Super  High  Frequency  (SHF)  This  band  extends 
from  3000MHz  to  30,000MHz  and  offers  the 
possibility  of  highly  directional  communication  links. 
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At  the  upper  end  of  the  band  a  1  metre  parabolic 
antenna  can  project  a  microwave  beam  with 
approximately  a  10  milliradian  (<1°)  divergence. 
Because  transmissions  can  be  isolated  geometrically 
as  well  as  by  frequency  allocation,  a  large  number  of 
video  transmissions  can  take  place  in  the  same  band 
without  necessarily  interfering  with  each  other  or 
with  other  kinds  of  transmissions.  Radiated  power 
levels  can  be  reduced  because  the  transmissions  are 
highly  directional.  The  signals  are  more  difficult  to 
detect  and  to  jam  because  they  are  so  tightly  focused. 
Also,  equipment  can  be  much  smaller  than  at  lower 
frequencies  because  the  wavelengths  are  so  much 
smaller.  The  principal  disadvantages  are  that:  (1) 
because  of  their  directionality,  relays  must  be  used  to 
circumvent  barriers,  (2)  they  are  less  able  to 
penetrate  buildings  and  foliage/canopy,  and  (3)  they 
are  susceptible  to  attenuation  by  rain  or  fog. 

Since  UGV's  will  carry  position  fixing  systems,  e.g. 
a  GPS  receiver,  to  provide  positional  information  at 
all  times,  and  since  the  locations  of  control  stations 
will  probably  be  fixed  and  known  a  priori,  the 
pointing  of  the  communications  antennae  can  be  done 
with  relative  ease. 

The  need  for  an  airborne  relay  to  allow  high 
frequency  directional  links  to  operate  out  of  line  of 
sight  ados  to  the  complexity  of  the  total  systems 
concept.  A  simple  balloon  mounted  transponder  is 
perhaps  the  simplest  solution.  The  balloon  would 
have  to  be  deployed  in  a  covert  manner  making  use 
of  ground  topography,  woods  or  buildings.  It  would 
not  need  to  be  flown  near  to  the  command  centre, 
making  this  vulnerable,  but  it  would  have  to 
guarantee  line  of  sight  both  to  the  command  centre 
and  to  the  UGV  throughout  the  whole  of  its  mission. 
An  unmanned  air  vehicle  (UAV)  is  another 
possibility,  but  there  is  an  incompatibility  between 
the  limited  endurance  time  for  the  UAV  to  be  on- 
station  (measured  in  hours)  and  the  long  duration 
UGV  missions  which  could  he  measured  in  days. 

Extremely  High  Frequency  (EHF)  This  millimetre 
wave  region  extends  from  30,000MHz  (30GHz)  to 
300,000MHz  (300GHz).  Many  of  the  remarks  made 
on  the  SHF  region  also  apply  to  EHF  but,  except  for 
windows  at  36GHz  and  94GHz,  atmospheric 
absorption  is  a  severe  problem,  limiting  operation  to 
short  range  applications.  The  advantages  of  smaller 
dimensioned  equipment  tend  to  be  offset  by  the 
expensive  circuit  technology  which  is  needed  at  these 
frequencies. 

Packet  Radio  So  far  we  have  considered  point  to 
point  radio  links  with  or  without  a  single  relay 
interspersed  between  transmitter  and  receiver.  The 
advent  of  packet  radio  allows  a  network  strategy  to 
he  adopted  which  may  have  several  advantages  in 
field  situations.  Serial  data  messages  from  a 
computer  are  attached  with  routing  addresses  and 
error  detection  codes  to  form  "packets"  of  data. 
Each  transceiver  in  the  network  is  configured  as  a 
node  which  allows  messages  to  be  forwarded  to  other 
nodes  onlv  if  they  are  received  without  errors.  The 
sender  of  the  message  needs  to  know  only  the 
nearest  node  which  he  has  to  contact.  This  node 
decides  on  the  path  to  send  the  message  and  if  one 
path  is  not  accessible,  it  will  automatically  try 
another  path  around  the  network.  This  can  be  very 
useful  in  a  conflict  situation  where  it  can  be  expected 


that  some  nodes  in  the  network  will  be  unavailable. 
Since  the  length  of  path  segments  can  be  reduced  by 
operating  through  intermediate  nodes,  the  frequency 
used  becomes  less  critical  as  the  link  distances  are 
shorter.  Alternatively,  the  individual  radiated  power 
from  each  node  can  be  less  than  if  no  intermediate 
nodes  are  used,  which  can  be  beneficial  in 
maintaining  covert  operation.  Inevitably  the  packet 
system,  with  user  access  at  9.6kbits/sec  and  carrier 
frequencies  in  the  VHF/UHF  bands,  is  not  ideally 
suited  to  real  time  closed  loop  control.  This  is 
because  of  delays  which  may  occur  in  clearing  the 
message  through  the  network  and  in  competing  with 
other  traffic.  However,  for  communicating  with 
semi-autonomous  vehicles  where  time  is  not  so 
critical,  the  "packet"  system  may  be  very  attractive. 

Satellites  The  earth  satellite  should  not  be  ignored 
as  an  extreme  form  of  airborne  relay. 
Geosynchronous  satellites  in  orbits  22,300  miles 
from  the  earth  would  impose  0.25  second  round  trip 
propagation  delays  and  would  hardly  be  suitable  for 
teleoperated  driving.  Low  earth  orbit  satellites  do 
not  suffer  this  disadvantage  but  a  constellation  of 
satellites  would  be  necessary  to  provide  continuous 
coverage,  and  separate  studies  are  needed  for 
particular  theatres  of  operation.  For  instance,  a  two 
satellite  system  would  be  sufficient  to  provide  1 00% 
coverage  of  all  points  which  are  more  than  57°  from 
the  equator,  whilst  a  1500km  6  -  satellite 

constellation  would  provide  90%  coverage  of  the 
SW  Asia  region.  Another  alternative  is  to  add  a 
transponder  to  the  Motorola  global  cellular  telephone 
satellites.  The  family  of  77  satellites  is  due  to  be  in 
position  by  1996.  Because  the  satellites  will  be  in 
low  overlapping  polar  orbits,  there  will  be  a  need  for 
continuing  replenishment  and  this  would  provide  the 
opportunity  to  add  a  military  module  or  a  secondary 
payload.  For  such  a  system  data  rates  will  be  at  a 
premium  with  2. 4kb its/sec  being  typical. 

Commercial  developments  are  revolutionising  the 
telephone  service  by  going  wideband  and  all  digital, 
and  low  data  rate  videophones  are  now  becoming 
available.  The  prospect  of  tclcoperating  equipment 
in  whatever  part  of  the  world  it  has  to  function 
becomes  a  possibility',  with  local  cellular  networks 
followed  by  global  satellite  links,  leaving  only  RF 
and/or  fibre  optic  links  for  the  last  few  kilometres  of 
the  transmission.  Even  today  there  is  scarcely  a  spot 
on  the  globe  which  is  not  within  CB  range  of  the 
global  telecommunications  net,  as  the  recent  Gulf 
War  has  demonstrated. 

7.2  Countermeasures 

There  will  be  a  need  to  make  the  UGV  and  the 
command  centre  with  its  service  crew  as  covert  as 
possible  in  order  to  achieve  the  battlefield  advantage 
of  removing  the  crew  from  hazardous  areas.  Again 
with  the  command  centre  transmitting  the  low  er  data 
rate  control  signals,  this  should  present  less  of  a 
problem  than  the  UGV,  which  can  be  expected  to  be 
physically  nearer  hostile  forces  and,  for 
teleoperation,  will  be  continuously  transmitting  video 
data.  Both  problems  are,  of  course,  eased  if  data  can 
be  transmitted  at  high  frequencies  in  focused  narrow 
beams,  and  the  data  traffic  can  be  reduced  to 
intermittent  exchanges  only,  by  the  introduction  of 
partial  autonomy. 
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With  electronic  surveillance  systems  now  being  able 
to  detect  and  locate  VHF  transmission  in  a  matter  of 
seconds,  attention  turns  to  frequency  hopping  and 
spread  spectrum  modes  to  give  a  low  probability  of 
interception.  Spread  spectrum  will  not  be  intercepted 
by  the  casual  listener  and  encryption/decryption  can 
be  used  to  improve  this  aspect.  As  well  as  these 
features,  spread  spectrum  can  provide  resistance  to 
jamming.  Frequency  hopping  involves  a 
conventionally  modulated  carrier  bemg  intermittently 
shifted  in  frequency  over  a  bandwidth  much  wider 
than  the  information  bandwidth.  Typically  the  dwell 
time  on  each  frequency  will  be  of  the  order  of  10  to 
400milliseconds.  It  was  developed  as  a  counter  to 
narrow  band  jamming  which  would  be  effective  if  a 
fixed  carrier  frequency  were  used.  However, 
because  the  peak  carrier  power  at  each  frequency  in 
the  hop  sequence  is  the  same  as  it  would  be  if  a 
single  fixed  carrier  frequency  were  used,  it  can  easily 
be  detected  by  wideband  surveillance  receivers.  In 
fact,  the  hopping  can  make  the  signal  easier  to  detect 
with  such  a  receiver  than  would  be  the  case  if  a 
single  carrier  frequency  were  used,  as  the  latter  can 
be  visually  masked  by  other  conventional  fixed 
frequency  signals  operating  around  the  same 
frequency. 

An  alternative  and  more  effective  approach  is  to  use 
Direct  Sequence  Spread  Spectrum.  This  achieves  a 
spreading  of  the  spectrum  by  modulating  the  original 
signal  with  a  very  wideband  signal  (the  chip 
frequency)  relative  to  the  data  bandwidth.  This 
wideband  signal  is  chosen  to  have  two  possible 
amplitudes  +1  and  -1,  and  these  amplitudes  are 
switched  in  a  pseudo-random  manner  periodically 
(Fig.  7).  Thus  at  each  equally  spaced  time  interval,  a 
decision  is  made  as  to  whether  the  wideband 
modulating  signal  should  be  +1  or  -1.  The  pseudo 
random  sequence  is  generated  electronically  and  is 
know  a  priori  to  the  transmitter  and  receiver.  The 
resulting  bit  stream  shown  in  Fig.  7  is  commonly 
modulated  on  to  the  carrier  by  using  180°  phase-shirt 
keying  (BPSK)  producing  a  wide-band,  suppressed- 
carrier  signal.  In  all  spread-spectrum  transmission 
systems,  the  receiver  has  to  identify  and  be  able  to 
lock  on  to  and  track  the  transmitted  pseudo-random 
sequences.  The  simplest  approach  is  to  examine  all 
possible  phase  relationships  between  the  internally 
generated  pseudo-random  code  and  the  received 
signal  until  a  match  is  found.  This  is  achieved  by 
allowing  the  two  signals  to  slip  past  each  other,  by 
offsetting  their  respective  clock  frequencies,  whilst 
continuously  correlating  the  two  signals.  Once  a 
correlation  peak  is  found,  a  code-tracking  loop  can 
be  brought  into  action,  which  will  synchronise  and 
track  the  two  signals.  As  an  example,  consider 
2.4kbits/sec  data,  which  is  BPSK  modulated.  The 
resulting  signal  bandwidth  is  4.8kHz.  This 
bandwidth  is  then  spread  using  direct  sequence 
spread  spectrum  to  15MHz  (the  chip  frequency')  on  a 
carrier  of  a  few  GHz.  This  gives  a  bandwidth  spread 
increase  of  15 MHz/4. 8kHz  or  3000:1  (or  35db) 
which  is  a  measure  of  the  suppression  achieved  on 
the  primary  data  signal.  It  should  be  pointed  out, 
though,  that  the  bandwidth  spread  increase  will  be 
limited  by  the  maximum  earner  frequency  that  can 
be  used,  which  in  turn  determines  the  chip 
frequency.  The  scope  for  application  of  the 
technique  will  be  limited  therefore  for  carrier 
frequencies  in  the  VHF  band.  Nevertheless,  the 
technique  offers  a  very  attractive  method  of 


suppressing  detectability  of  the  command  centre  and 
of  providing  jamming  resistance. 

7.3  Fibre  Optics 

Many  of  the  problems  associated  with  radio 
transmission  are  totally  avoided  by  the  use  of  fibre 
optic  communications  systems.  *  Several  video 
channels  can  be  accommodated  on  a  single  fibre,  full 
duplex  operation  is  possible  over  the  same  fibre 
system,  the  source  of  transmission  is  not  detectable, 
and  there  is  freedom  from  mutual  interference 
between  channels  of  friendly  forces.  Their  low 
weight,  high  bandwidth,  and  low  transmission  loss 
make  them  attractive  by  comparison  with  wire-cored 
cable  for  distances  of  up  to  many  kilometres. 
Against  these  favourable  features,  however,  must  be 
weighed  the  operational  acceptability  of  reliance  on  a 
permanent  and  probably  surface  laid  cable,  whose 
integrity  must  be  questioned  for  long  distances,  and 
over  long  periods  of  time.  Although  cables  have 
been  demonstrated  to  withstand  being  driven  over  by 
armoured  vehicles,  showing  that  very  robust 
protection  can  be  provided  by  suitable  cladding 
materials,  there  is  always  the  chance  of  snagging  or 
fracturing  arising  at  some  point  in  the  path  This 
would  clearly  render  the  UGV  inoperative  unless  a 
back  up  system  were  provided  to  cover  such  an 
eventuality.  A  suitable  back  up  facility  might  well 
be  supplied  by  a  radio  system  where,  as  a  secondary 
or  emergency  link,  its  other  limitations  may  make  it 
acceptable  in  this  role.  The  question  of  cost  cannot 
be  neglected,  since  at  about  £1  per  metre  the 
dispensing  of  many  kilometres  of  cable  is  an 
expensive  item  unless  the  cable  can  be  recovered  and 
used  for  further  missions.  Cable  is  available  in  two 
forms.  One  is  a  well  protected  version  which  can  be 
gathered  up  after  a  mission,  rewound  and  then 
reused.  A  lighter  weight  version  is  prewound  on  a 
spool  dispenser,  wax-impregnated  to  provide 
controllable  payout  dynamics,  and  is  for  single 
operation,  single  mission,  only.  Again,  there  is  the 
option  of  using  the  durable  version  for  trials  and 
experimental  use,  whilst  the  lighter  weight  cable 
could  be  retained  for  operational  use.  Portable 
equipment  is  available  for  carrying  out  repairs  to 
cable  in  the  field.  Fractures  can  be  located  by  using 
a  laser  rangefinder  to  probe  the  fibre  along  its  length 
and  reflect  off  the  discontinuity  caused  by  the 
fracture.  Once  found,  the  damaged  fibre  ends  are 
exposed  and  removed  by  controlled  cleaving.  The 
two  cleaved  surfaces  are  then  aligned  in  a  fusion 

Sheer  and  electric  current  is  used  to  heat  and  fuse 
s  adjacent  surfaces.  A  plastic  heat  shrink  with 
metal  strengthening  is  then  used  to  support  the 
repaired  fibre. 

Trials  of  UGVs  have  been  carried  out  very 
successfully  over  a  wide  range  of  conditions  using  a 
fibre  optic  tether  as  the  primary  communications 
link,  and  further  details  or  system  options  deserve 
description. 

The  first  choice  is  between  single  and  multimode 
optical  fibre.  The  main  tradeoff  has  been  optical 
performance  versus  ease  of  connection.  The  high 
performance  single  mode  fibre  has  a  small  optical 
core  (~  9  microns)  set  in  a  125  micron  diameter 
silica  sheath,  and  connectorisation  requires  precise 
alignment  to  achieve  acceptable  loss.  Multimode 
fibres,  on  the  other  hand,  generally  have  50-62.5 
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micron  core  diameters,  again  within  a  125  micron 
diameter  outer  sheath.  The  optical  performance  is 
defined  primarily  by  attenuation  and  bandwidth. 

Attenuation  The  attenuation  of  optical  fibre  is  very 
near  the  theoretical  limit  imposed  by  Rayleigh 
scattering,  the  spectral  curve  generally  following  an 
inverse  relationship  with  (wavelength)4.  Absorption 
therefore  is  practically  zero  over  the  wavelength 
ranee  850  to  1550  nanometres  where  the  best  system 
performance  (transmitter/fibre/receiver)  obtains. 
With  attenuation  as  low  as  0.2db/km  single  mode 
cables  can  operate  without  repeaters  at  distances  up 
to  300km  between  source  ana  detector  depending  on 
the  data  rate  bandwidth  requirements  (Ref.5). 

Bandwidth  The  bandwidth  limitations  of  a  single¬ 
mode  fibre-optic  system  arise  from  dispersion.  A 
pulse  travelling  down  the  fibre  will  broaden  due  to 
different  spectral  components  of  the  optical  source 
having  different  transmission  speeds  in  the  fibre.  To 
maximise  bandwidth  therefore,  we  need  to  select 
narrow  bandwidth  sources.  Low  cost  edge  lasers 
(In  GaAsP)  have  a  spectral  width  of  2-5  nanometres 
compared  with  80-120  nm  for  the  high  performance 
Light  Emitting  Diodes  (LED)  which  are  used  as  a 
source  for  multimode  fibres.  Standard  pure  silica 
fibre  has  a  zero  dispersion  wavelength  close  to 
1300nm  and  hence  narrow  spectral  sources  with 

Kiak  power  at  this  wavelength  should  be  chosen. 

owever,  it  is  possible  to  tailor  the  material  of  the 
fibre  core  to  give  a  zero  value  of  dispersion 
(picoseconds  per  nm  per  km)  at  specific  frequencies 
up  to  1550  nm  (dispersion  shifted  fibres). 
Multimode  fibres  have  an  additional  pulse 
broadening  effect  due  to  different  optical  paths  (or 
modes)  within  the  fibre  having  different  transit  times, 
and  this  greatly  reduces  available  bandwidth 
compared  with  single  mode  fibres.  Multimode  fibres 
(50/125)  have  (modal  bandwidth  x  km)  products 
typically  less  than  lOOOMHz.km.  For  a  teleoperated 
multi-video  system  therefore  it  can  be  seen  that  a 
multimode  fibre  optic  system  will  limit  the  cable 
distance  to  around  10  km  or  a  maximum  operational 
range  of  5km  if  recovery  is  intended. 

Duplex  Link  with  Single  Fibre  A  single  fibre  cable 
allows  the  simultaneous  transmission  of  optical 
signals  in  opposite  directions.  This  can  be  achieved 
by  wavelength  multiplexing  or  directional  couplers. 
Wavelength  multiplexers  enable  the  transmission  of 
two  signals,  each  with  a  different  wavelength,  to  be 
combined  on  to  a  single  fibre.  In  this  way,  using 
1300nm  and  1500nm,  or  850nm  and  1300nm 
sources,  for  example,  a  duplex  link  can  be  set  up.  A 
single  wavelength  single  fibre  duplex  link  can  be 
constructed  using  2-way  directional  couplers.  These 
are  passive  and  nave  the  property  that  power  is  split 
between  2  fibres  in  a  predetermined  ratio,  nonnally 
50:50.  The  disadvantage  is  that  5db  of  additional 
loss  is  introduced  ana  reflected  power  with  the 
system  leads  to  increased  noise. 

Signal  Multiplexing  and  Modulation  Methods  used 
in  conventional  electrical  systems  can  equally  well  be 
applied  to  each  optical  channel.  These  include  digital 
Time  Division  Multiplexing  (TDM)  and  subcarrier 
Frequency  Division  Multiplexing  (FDM).  FDM  is 
suitable  for  applications  involving  multi-analogue 
video  cameras.  Amplitude  or  frequency  modulation 
of  subcarriers  requires  only  simple  analogue 


modulators  followed  by  a  combiner  or  mixer.  The 
broadband  signal  is  then  used  to  amplitude  modulate 
the  laser  or  LED  source.  Often  digital  data  channels 
have  to  be  multiplexed  with  other  analogue  channels. 
In  a  subcarrier  FDM  scheme  carrying  video 
bandwidth  data  channels,  digital  data  can  be  used  to 
modulate  a  separate  video  subcarrier.  It  is  the  top 
end  sub-earner  frequency  and  its  modulation 
products  which  set  the  bandwidth  requirements  for 
the  optical  link,  and  need  to  be  taken  into  account  in 
the  overall  link  design. 

We  have  seen  that  the  optical  performance  of  a  single 
mode  fibre  optic  system  will  place  little  restriction  on 
the  communication  requirements  for  teleoperation. 
But  die  physical  size  and  robustness  are  more 
questionable.  For  comparison,  a  dispenser  coil  to 
accommodate  20km  of  fibre  will  weigh  5kg  for 
0.5mm  diameter  fibre,  or  34.4kg  for  1.5mm  diameter 
(Ref.5).  The  0.5mm  diameter  fibre  is  for 
applications  where  working  tensions  are  low  (<lkg), 
but  when  these  are  higher,  the  fibre  is  reinforced  by 
poly-aramid  yam  and  results  in  the  diameter 
increasing  to  1  to  1.5mm.  Having  installed  the 
dispenser  coil  on  board  the  remote  vehicle  and 
subsequently  dispensed  the  fibre,  the  question  of 
protection  of  the  surface  laid  fibre  remains.  In  some 
terrain  situations,  it  is  possible  to  cut  a  narrow 
furrow  as  part  of  the  dispensing  mechanism  and  then 
lay  the  fibre  within  the  furrow,  even  compacting  it 
afterwards  for  increased  security.  The  other  problem 
to  be  overcome  is  the  manipulation  of  fibre  during 
vehicle  maneouvres,  especially  in  confined  spaces, 
when  the  possibility  of  self-snagging  becomes 
serious. 

8.  LOW  BANDWIDTH  TELEOPERATION 

We  have  indicated  the  need  to  reduce  the  bandwidth 
of  the  imaging  data  fed  back  from  the  UGV  to  the 
Command  Centre  in  order  to  ease  the  requirement  on 
the  communications  system.  Ideally  this  should  be 
compatible  with  the  control  data  rate  and  16kbits/sec 
would  appear  to  be  a  significant  target  to  aim  for.  A 
full  frame  video  image  is  defined  fry  gray  level  (8 
bits),  pixel  number  (6252)  and  frame  rate  (25/s) 
giving  78Mbits/sec.  To  reduce  this  to  16kbits/sec 
would  require  a  compression  ratio  of  4900. 

Clearly  substantial  reductions  in  bit  rate  (4  bits), 
pixel  number  (1282)  and  frame  rate  (10/s)  could  give 
a  compression  ratio  of  125  without  anticipating  a 
major  impact  on  performance.  But  there  is  still  a 
shortfall  of  x40  in  the  compression  ratio  required. 
Trials  have  been  conducted  both  at  fiillscale  and  in 
simulation  to  test  the  sensitivity  to  frame  rate  and 
pixel  number  (resolution)  in  particular.  Broadly  the 
speed  at  which  the  UGV  can  be  driven  is  dependent 
on  the  frame  rate  in  the  range  25/s  to  1/s  as  this 
determines  the  ability  of  the  driver  to  control  the 
vehicle  accurately  and  to  trace  the  route  ahead.  In 
one  particular  trial  over  unfamiliar  cross  country 
terrain,  a  maximum  speed  of  45kph  achieved  at  a 
frame  rate  of  25/s  was  reduced  to  25kph  at  8/s. 
Reducing  the  pixel  number  from  6252  to  1282  was 
not  very  significant,  although  it  increased  the  stress 
on  the  driver  in  looking  out  for  obstacles  in  the  path. 
On  ihe  other  hand,  the  pixel  number  could  be 
reduced,  whilst  preserving  the  resolution,  by 
presenting  the  image  in  "letter  box"  format,  rather 
than  the  full  frame  picture,  by  dispensing  with  the 
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upper  part  of  the  image  which  is  of  little  value  for 
driving  purposes.  This  exhausts  the  relatively  simple 
techniques  for  increasing  the  compression  ratio,  and 
we  must  now  examine  how  image  processing  might 
be  applied  to  make  further  improvements. 

Techniques  may  be  described  under  two  main 
headings  (i)  intra  frame  processing  and  (2)  image 
sequence  processing.  The  intraframe  approach  can 
be  tackled  either  from  a  spatial  or  frequency  domain 
viewpoint  (Ref.  6). 

Intraframe  Spatial  Processing  (a)  Predictive 
coding  (DPCM).  This  approach  has  been  studied  for 
the  past  40  years,  ana  relies  on  a  prediction  of  a 
particular  "element"  being  made  on  the  basis  of 
nearby  picture  elements.  The  difference  signal  is 
then  efficiently  quantised  and  coded.  It  is  possible  by 
developing  algorithms  of  this  type  to  achieve 
1  bit/clement. 

(b)  Vector  Quantisation  tVQ).  In  this  case  the  image 
field  is  divided  into  small  blocks  and  each  is,  in  turn, 
compared  with  similar  entries  in  a  codebook.  For  the 
closest  match,  an  index  corresponding  to  that  entry  is 
sent  to  the  receiver  (which  also  has  access  to  the 
codebook)  which  then  uses  the  same  codebook  entry 
for  reconstruction.  Problems  hinge  on  the  adequacy, 
the  construction  and  search  of  the  codcbook. 

(c)  Segmented  Coding.  Many  simpler  pictures  can 
be  considered  to  consist  of  areas  of  slowly  varying 
luminance,  colour  or  texture,  separated  by  "well 
defined  edge  detail.  The  technique  involves 
segmenting  the  image  and  then  representing  the  detail 
with  suitable  approximating  functions.  It  is  the 
accurate  coding  of  the  boundaries  which  presents  a 
problem  here. 

(d)  Pyramid  and  Wavelet  Coding.  This  involves  the 
decomposition  of  image  detai  l  into  sub  images  having 
graded  resolution.  Hie  original  image  is  low-pass 
filtered  to  generate  a  reduced  resolution  image  at  the 
next  level,  and  this  operation  is  repeated  as  desired. 
This  structure  is  the  "Gaussian"  pyramid  and  its 
function  is  to  act  as  a  prediction  for  the  previous 
higher  resolution  step.  More  recently  there  has  been 
the  introduction  of  "wavelet"  structures  having 
localised  spatial  and  frequency  domain  properties. 

(e)  Fractals.  This  has  affinities  with  VQ  but  an 
important  difference  is  that  the  equivalent  of  tire  VQ 
codebook  need  not  be  known  to  the  decoder.  Results 
seem  to  indicate  ■%  bit/clement  to  be  achievable. 


number  of  separate  frequency  bands  for  coding  and 
transmission.  The  reconstructions  at  the  receiver  are 
then  summed  to  produce  the  output  image.  Sub-band 
coding  has  about  the  same  performance  as 
Transform  Coding  and  is  well  suited  to  progressive 
transmission  and  multi-resolution  applications. 

Image  Sequence  Processing  The  dimension  of  time 
provides  another  means  of  coding  compression.  If 
only  the  changes  in  the  interframe  seauence  are 
transmitted  then  considerable  savings  in  data  traffic 
can  be  made.  This  is  particularly  effective  in  the 
case  of  teleoperated  driving  where  changes  occur 
mostly  in  the  foreground  w  here  the  computer  power 
needs" to  be  deployed,  rather  than  in  the  upper  part  of 
the  image,  covering  the  more  distant  scene  which  is 
relatively  invariable. 

(a)  Three  dimensional  techniques.  These  can  be 
extended  to  two  dimensional  predictive  coding, 
transform  or  sub-band  coding.  Multiple  frames  are 
stored  and  an  adaptive  switched  predictor  operates 
on  the  basis  of  local  image  activity. 

(b)  Hybrid  Coding.  Here  an  interframe  prediction 
step  produces  an  error  image  frame  w'hich  is  then 
subjected  to  further  (spatial)  processing. 

(c)  Model  Based  Coding.  Image  analysis  techniques 
can  yield  parameters  which  when  transmitted  to  the 
decoder,  can  be  used  for  the  resynthesis  of  the 
image.  Given  a  particularly  constrained  image  scene 
such  as  a  road  or  track,  a  "Wire  Frame"  model,  used 
in  conjunction  with  texture  mapping,  might  allow  an 
acceptable  representation.  Location  of,  and  tracking 
information  on,  changing  features  are  sent  to  the 
receiver  as  updates.  The  transmission  rate  may  need 
to  be  only  a  few  hundred  bits/sec  depending,  of 
course,  on  the  number  and  rate  of  the  changing 
features. 

9.  CODING  TECHNIQUES 

For  any  data  transmitted  over  a  communication  link 
there  is  the  possibility  of  corruption  due  to  noise  or, 
in  the  case  of  a  radio  link,  interference  or  fading. 
Coding  techniques  will  be  required  to  provide  error 
detection  and  correction  to  ensure  that  the  receiver 
deals  only  with  uncorrupted  data.  This  will  be 
particularly  important  for  low'  bandwidth  systems 
where  the  degree  of  redundancy  in  the  data  is 
minimal. 

Fig.  8  illustrates  a  simplified  communication  system. 
The  source  (transmitter)  sends  a  message,  via  the 


Intraframe  Frequency  Domain  Processing 

(a)  Transform  Coding.  'The  spectra  of  most 
"natural"  images  are  strongly  low  pass,  and  a 
suitable  transform  such  as  the  Direct  Cosine 
Transform  DCT,  will  therefore  produce  a  set  of 
coefficients  where  high  " frequency terms  are  usually 
small  or  even  zero.  Significant  coefficients  are 
scaled,  quantised  and  transmitter!,  and  an  inverse 
transformation  at  the  decoder  reproduces  an 
approximation  to  the  input.  If  extreme  compression 
is  attempted,  block  structure  artefacts  appear  and 
this  adds  to  the  stress  on  the  operator. 

(b)  Sub-band  Coding.  This  has  a  close  affinity'  with 
the  wavelet  techniques.  The  image  is  split  into  a 


SOURCE  SINK 
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Fig. 8  Simple  communication  system 
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encoder,  across  the  channel,  where  the  encoded 
message  might  be  corrupted  by  channel  noise.  The 
message  then  passes  into  the  decoder  where, 
hopefully,  the  corruption  due  to  channel  noise  can  be 
eliminated,  and  the  correct  message  passed  to  the 
sink  (receiver). 

The  decoder  takes  the  received  word  and  compares  it 
to  every  acceptable  codeword  to  find  the  "distance" 
between  them.  The  "distance"  is  the  number  of 
places  (bits)  in  which  two  words  differ.  It  finds  the 
minimum  distance  between  the  received  word  and  a 
codeword.  If  there  is  only  one  codeword  which  is 
this  distance  from  the  received  word,  then  the 
received  word  is  decoded  as  this  codeword.  If  there 
is  more  than  one  codeword  this  distance  away,  the 
decoder  cannot  decode  the  received  word.  The 
minimum  distance  between  codewords  'd'  gives  the 
code's  error  correcting/detecting  ability.  Simple 
coding  theory  shows  that  the  number  or  detectable 
errors  is  (d  -  1)  and  the  maximum  number  of 
correctable  errors  is  <(d  -l)/2.  Thus  a  code  word  of 
only  5  bits,  with  a  minumum  distance  of  4,  can 
correct  one  error  and  detect  up  to  3  errors. 

Coding  theory  has  generated  various  types  of  codes 
each  effective  under  different  conditions.  The  choice 
of  code  therefore  will  be  very  dependent  on  the  type 
of  channel  noise  expected. 

10.  POSITION  FIXING 

Before  leaving  teleoperation  it  is  worth  reviewing  the 
role  of  position  fixing  in  the  control  of  the  remote 
vehicle.  So  far  it  has  been  sufficient  for  the  operator 
to  have  position  data  supplied  on  line  to  him  for  the 
purpose  of  assisting  him  in  recognising  waypoints  en 
route,  and  in  determining  "distance  to  go"  to  the 
destination.  In  general,  whilst  knowledge  of  the 
remote  vehicle  position  is  essential  to  the  operator  to 
avoid  the  vehicle  becoming  lost,  the  accuracy  need 
only  be  sufficient  for  this  purpose.  It  will  be 
determined  by  the  quality  of  map  information 
available,  and  the  density  and  visibility  of  waypoints. 
Heading  to  0.5  degree  and  position  to  100m  may  be 
acceptable  for  this  "advisory"  role.  This  may  be 
achieved  by  gyroscopically  based  inertial  navigation 
equipment,  but  a  less  expensive  dead  reckoning 
system  using  a  heading  reference  and  odometer 
sensing  may  be  adequate  for  shorter  range  vehicles, 
where  the  operator  can  zero  the  on  board  system 
when  he  arrives  at  recognisable  waypoints.  The  key 
point  in  the  teleoperation  mode  is  that  the  position 
fixing  sensor  is  not  working  within  the  guidance  and 
control  loop  as  is  the  case  with  supervisory'  and 
autonomous  control.  In  these  latter  modes  of 
operation,  the  vehicle  position  accuracy  will  be 
determined  by  the  achievement  of  success  or  failure 
in  the  mission,  measured  in  terms  of  the  vehicle 
position.  In  other  words,  it  should  be  commensurate 
with  the  ability  of  a  manned  driver  to  position  the 
vehicle  safely,  for  example,  to  prevent  the  vehicle 
accidentally  leaving  the  road  and  ending  up  in  a 
ditch.  In  relative  terms,  therefore,  with  reference  to 
the  actual  position  of  the  vehicle,  obstacles  will  need 
to  be  located  to  perhaps  10cm.  In  absolute  terms, 
i.e.  with  reference  to  tne  local  mapping  information, 
accuracies  of  l-20metres  will  be  required  if 
significant  errors  in  navigation  are  to  be  avoided. 


Global  Positioning  Systems  The  advent  of  the 
Global  Positioning  System  (known  as  GPS)  has 
provided  the  basis  for  a  highly  accurate  system 
capable  of  measuring  the  position  of  any  point  on  the 
earth's  surface  to  within  a  few  centimetres.  Based  on 
6  constellations,  each  with  3  satellites,  in  different 
orbit  planes  around  the  earth  (the  expensive  part  of 
the  system),  a  receiver  on  the  user  vehicle  becomes  a 
small  and  relatively  inexpensive  unit  which  can 
nevertheless  extract  the  positional  data  at  incredible 
precision.  (Ref.  7)  The  satellites  all  transmit 
continuously  on  the  same  frequency  of  1575MHz 
and  a  method  of  separation  and  identification  is 
required,  as  data  from  3  satellites  have  to  be 
received  to  give  a  2D  fix  and  from  4  satellites  for  a 
3D  fix.  In  addition,  the  carrier  frequency  chosen 
gives  rise  to  an  ambiguity  every  20cm.  The 
application  of  spread  spectrum  modulation  (Fig.7) 
solves  these  three  problems  simultaneously.  The 
code  (the  C/A  code)  runs  at  1 -023MHz  and  has  a 
length  of  1023  bits.  By  giving  each  satellite  its  own 
code,  many  satellites  can  co-exist  without 
interference  and  each  may  be  individually  identified. 
In  addition,  the  satellites'  own  position  and 
transmission  times  have  to  be  known  since  the 
positional  information  in  each  plane  relies  on  time 
delay  ranging  between  two  satellites  at  a  time.  This 
information  is  provided  by  a  further  level  of 
modulation.  Correction  to  the  satellite  position  to 
allow  for  transmission  delays  is  handled  by  each 
satellite  transmitting  the  equations  of  motion  of  its 
orbit.  In  addition  to  the  C/A  code,  each  satellite 
transmits  a  precise  code,  known  as  the  P  code,  which 
is  reserved  for  the  military.  This  operates  at  a 
chipping  rate  ten  times  faster  than  the  C/A  code 
giving  a  ten  times  wider  bandwidth  and  therefore 
greater  jamming  resistance  (see  Section  7.2).  The 
P  code  signal  can  also  be  changed  to  another  code  in 
times  of  emergency  to  give  antispoofing  protection. 
From  the  user  viewpoint  there  remain  two 
difficulties.  Firstly,  the  availability  of  sufficient 
satellites  at  an  elevation  which  enables  the  high 
accuracy  to  be  maintained.  This  will  be  optimised 
when  the  target  number  of  satellites,  namely  21,  is 
finally  placed  in  orbit  in  1995.  Secondly,  the 
visibility  of  satellites  to  the  UGV  due  to  the  screening 
effects  of  buildings,  trees  etc.  These  effects  are 
likely  to  force  tne  user  to  fit  some  independent 
position  fixing  system  to  cover  the  situations  when 
the  GPS  is  not  available  to  the  UGV. 

Whilst  GPS  is  basically  a  position  fixing  system, 
recent  developments  using  multiple  antennae  on  a 
round  vehicle,  have  enabled  heading  information  to 
e  derived  when  the  vehicle  is  moving  or  stationary. 
By  mounting  antennae  at  a  spacing  of,  typically, 
1* metre  azimuth  accuracies  of  0.2-0. 4°  have  been 
obtained. 

11  SUPERVISORY  CONTROL 

We  have  defined  this  mode  of  control  as  intermediary 
between  teleoperation  and  autonomous  control.  It 
assumes  the  man  is  not  an  integral  part  of  the  control 
loop,  and  that  he  provides  only  intermittent 
information  on  new  waypoints  or  alternative  routes 
for  the  UGV.  The  essential  element  is  that  the 
vehicle  should  be  able  to  implement  an  unintelligent 
path  to  a  distant  wavpoint  whilst  travelling  at  a 
reasonable  speed  ana  arriving  with  acceptable 
positional  accuracy.  In  the  first  instance  we  shall 
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assume  that  the  terrain  is  reasonably  flat  and 
obstacle  free.  The  MARDI  system,  described  in 
Section  3,  was  adapted  to  demonstrate  the  principle 
of  supervisory  control  in  a  cost  effective  way.  To 
avoid  hardware  changes  in  the  remote  vehicle  itself,  a 
single  supervisory  control  unit  was  designed  as  an 
interface  unit  installed  in  the  Command  Centre  where 
it  communicates  with  the  Main  Computer  through 
shared  memory.  The  Commander  enters  future 
waypoints  as  icons  on  his  map  display,  which  also 
presents  the  current  position  of  tne  UGV  via  the 
Feedback  data  from  the  inertial  navigation  equipment. 
The  Supervisory  Control  Unit  then  translates  the 
path  to  the  first  waypoint  into  a  series  of  heading  and 
speed  demands  which  are  transmitted  along  the 
communications  link  to  the  UGV,  just  as  if  a  human 
operator  were  physically  manipulating  the  control 
functions  for  normal  telcopcration.  In  order  to  do 
this,  of  course,  the  Supervisor}/'  Control  Unit  has  to 
have  built  in  a  model  or  the  vehicle  control  system  to 
ensure  that  the  vehicle  is  not  asked  to  carry  out 
impossible  demands,  and  yet  will  perform  the 
necessary  maneouvres  by  making  optimum  use  of 
steering, ’throttle,  brakes  and  gearbox  controls. 

The  system  was  exercised  in  a  field  environment  and 
successfiilly  demonstrated  that  it  could  reach  two 
waypoints  ’50- 100m  apart,  the  second  requiring  a 
heading  change  of  90b  from  the  first.  This  was 
carried  out  using  the  appropriate  gear  selections  and 
with  speeds  up  to  lOkph.  The  success  of  the 
Supervisory  Control  Unit  design  would  naturally 
lead  to  its  removal  from  the  Command  Centre  and 
the  incorporation  of  a  similar  system  in  the  UGV 
itself,  thus  reducing  traffic  on  the  communications 
link  to  the  Command  Centre  to  tell-back  data  and 
position  coordinates  of  the  waypoints  alone.  It 
should  be  emphasised  that  the  above  only  refers  to 
"blind"  driving.  It  has  taken  no  account  of  land 
topography  or  obstacles  and  simply  attempts  to 
implement  a  straight  line  path  between  designated 
waypoints.  Adding  an  obstacle  detection  capability 
would  then  require  an  automatic  means  of  obstacle 
negotiation  and  this  leads  logically  into  full}’ 
autonomous  control.  If  we  assume,  for  the  purposes 
of  Supervisory  Control,  that  the  human  operator 
takes  responsibility  for  route  finding,  waypoint 
marking,  and  obstacle  detection,  then  some  low- 
bandwidth  video  display  would  be  suitable  for  these 
functions  to  be  performed.  This  was  the  basis  for  the 
first  Computer  Aided  Remote  Driving  (CARD) 
system  developed  by  JPL  in  the  US.  Tne  operator 
designated  a  route  on  his  video  screen  and  this  route 
was  transformed  to  the  ground  plane  (assuming  a  flat 
ground)  and  then  transmitted  to  the  UGV  For  the 
vehicle  autopilot  to  implement.  Route  planning  was 
essentially  carried  out  on  a  still  frame,  although 
stereo  cameras  were  fitted  to  the  vehicle  to  give  some 
range  perception,  but  the  concept  was  essentially  a 
"stop-go"  routine.  The  most  distant  point  would  be 
limited  by  the  clarity  of  the  video  displayed  image 
and  accuracy  would  suffer  at  these  limits  of 
visibility. 

Feed  Back  Limited  Control  System  (FELICS) 

An  alternative  approach,  demonstrated  by  Dynamic 
System  Technologies  Inc.  in  the  US  in  1992, 
attempts  to  optimise  the  speed  at  which  a  manually 
designated  route  can  be  negotiated.  The  operator  has 
control  of  a  cursor  or  "puck"  which  can  be  moved 
around  over  the  video  display  generated  by  the 


camera  which  is  pan  and  tilt  mounted  on  the  UGV. 
This  camera  is  slaved  to  point  in  the  direction  of  the 
"puck".  The  "puck"  is  moved  by  the  operator  to 
trace  the  most  advantageous  route  towards  the 
eventual  destination,  and  as  it  does  so  it  spawns 
waypoints  along  the  route  at  1  metre  ground  plane 
spacing.  The  UGV  now  drives  itself  automatically 
through  these  waypoints  in  turn,  following  at  a  safe 
distance,  i.e.  an  appropriate  stopping  distance, 
behind  the  "puck".  If  route  planning  for  the  "puck" 
becomes  difficult,  the  Operator  switches  the  camera 
to  panning  mode  and  this  automatically  inhibits  the 
spawning  of  further  w'aypoints,  the  UGV  coming  to  a 
halt  at  the  last  designated  waypoint.  Successful 
navigation  was  achieved  with  the  system  operating  at 
1  frame/scc  on  a  300kbits/sec  data  link  and  operation 
at  1  frame  in  3.5sec  was  also  demonstrated.  The 
UGV  camera  was  mounted  high  on  the  vehicle  to 
give  a  good  depression  view  of  the  path  ahead,  but 
nevertheless,  the  number  of  waypoints  on  the  screen 
was  never  greater  than  about  ten  which  implied  a 
relatively  short  "look  ahead"  distance  of  10m.  This 
tends  to  confirm  that  the  degraded  video  display 
associated  with  the  low-  bandwidth  teleoperation  will 
limit  the  ability  of  the  human  operator  to  plan  routes 
ahead  over  significant  distances  within  his  field  of 
vision. 

When  the  vehicle  autopilot  takes  sole  responsibility' 
for  navigating  between  waypoints,  it  is  important 
that  the  designated  step  length  equals  the  actual 
distance  to  be  traversed  over  the  ground.  Normally  a 
flat  ground  plane  is  assumed,  but  ground  slope 
variations  between  the  local  vehicle  position  and  the 
designated  waypoint  position  will  lead  to  significant 
errors  in  the  demanded  step  distance.  Ideally,  this 
can  be  measured  directly  and  eliminated  using  a 
boresighted  laser  rangefinder  to  measure  the  exact 
distance  to  the  designated  waypoint,  but  the  group  at 
Carnegie  Mellon  University  in  the  US  is  investigating 
alternative  ways  of  reducing  the  errors  arising  from 
this  source,  by  measuring  the  local  ground 
inclination  of  the  vehicle  with  the  on  board 
navigation  system  at  the  latest  possible  time,  and 
using  this  to  correct  the  position  of  the  next 
waypoint.  This  they  have  termed  the  STRIPE 
technique  (Ref.  15). 

12.  AUTONOMOUS  CONTROL 

In  the  supervisory  mode  of  operation,  we  have 
assumed  that  man  is  available  to  deal  with  the  more 
difficult  route  planning  and  image  understanding 
tasks,  and  systems  have  been  described  which  show 
that  such  systems  can  already  be  built  and 
demonstrated  in  a  field  environment.  The  ultimate 
objective  scientifically  is  to  show  how  these  tasks  can 
be  carried  out  by  machine  intelligence,  and  we  need 
to  review  the  current  state  of  technology  and 
philosophy  in  order  to  establish  the  boundaries  of 
current  achievement,  and  our  expectations  for  the 
future  in  this  complex  and  challenging  area. 

12.1  System  Architectures 

Fundamental  to  our  understanding  of  the  problem  is 
our  ability  to  represent  the  operation  of  an  intelligent 
autonomous  vehicle  in  some  logical  and  structured 
form.  There  are  essentially  two  forms  of  architecture 
in  use  or  in  development  at  the  present  time.  Firstly, 
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Fig.9  Functional  Decomposition 


there  is  the  Functional  Decomposition  (Fig.9)  in 
which  the  robot  is  considered  as  a  group  of 
processes,  known  as  agents,  each  carrying  out  a 
specific  function.  The  agents  are  usually  the 
perception  system,  the  planner,  the  navigator  and 
pilot,  and  the  executor  or  controller.  The  second 
structure  is  the  Behaviourist  Decomposition 
(subsumption  architecture)  (Fig.  10)  which  considers 
the  robot  to  be  composed  or  levels  of  behaviour 
(avoid,  wander,  explore  etc.)  such  that  there  is  only 
performance  degradation  instead  of  total  system 
failure  when  one  of  the  higher  levels  fails. 
Additional  layers  of  intelligent  behaviour  may  be 
added  later  increasing  the  overall  "competence"  of 
the  system  (Ref. 8). 

NASREM  Functional  Architecture  The  US 
National  Institute  of  Standards  and  Technology 
(NIST)  has  attempted  to  generate  an  open  system 
(platform  independent)  control  architecture  for  real 
tune  advanced  autonomous  robotic  systems  that  is 
functionally  based,  and  which  incorporates  many  of 
the  features  of  current  research  projects  (Ref.9). 


Fig. 10  Behavioural  Decomposition 

This  has  developed  into  the  National  Standard 
Reference  Model  Architecture  (NASREM)  and  is 
shown  in  Fig.  11.  It  is  hierarchically  structured  with 
six  layers  each  performing  a  different  fundamental 
transformation  on  data.  It  is  also  partitioned 
horizontally'  into  three  sections,  task  decomposition, 
world  modelling  and  sensory  processing.  The  flow 
of  commands  and  status  feedback  is  hierarchical. 
Goals  are  decomposed  both  spatially  and  temporally 
throughout  the  different  levels,  with  each  task 
decomposition  node  on  a  given  level  receiving  input 
commands  from  only  one  higher  level  supervisor 
node  and  outputting  to  a  set  of  lower  level  nodes. 
Data  is  shared  horizontally  between  modules,  with 
much  higher  horizontal  data  rates  than  vertical  data 
rates.  The  architecture  is  structured  to  allow 
evolutionary  growth  in  capability  and  autonomy, 
with  comprehensive  human-computer  interaction 
(HCI)  acting  through  a  global  memory  management 
system.  The  six  general  levels  of  functionality  are 
now  described,  starting  at  the  lowest  level.  Level  1  - 
Actuator  or  Servo.  This  level  transforms  any 
motions  and  positions  expressed  in  a  generalised 
coordinate  system  for  the  mission,  e  g.  the  start 
point,  into  coordinate  systems  more  relevant  to 
particular  vehicle  sub-systems,  and  sends 

appropriate  control  signals  to  the  vehicle  effectors 
[Sample  rate  1kHz,  replan  time  1  msj.  Level  2  - 
Primitive  or  Component.  This  level  computes 
vehicle  dynamics  with  respect  to  an  appropriate 
generalised  coordinate  system  and  sends  vehicle 
control  commands  to  ensure  that  the  desired 
trajectory  within  this  coordinate  svstem  is  followed 
[Sample  rate  62Hz,  replan  time  16ms].  Level  3  - 
Elementary  (E)  Move/Subsystem.  This  level 

performs  a  piecewise  decomposition  of  elementary 
move  commands  into  a  series  of  intermediate  motions 
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Fig. 11  The  NASREM  Hierarchical  Architecture 
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and  positions.  These  intermediate  motion  commands 
define  a  path  which  has  been  checked  and  is  clear  of 
potential  obstacles,  and  can  be  physically 
implemented  by  the  vehicle  (Sample  rate  8Hz,  replan 
time  128ms].  Level  4  -  Task/Robot  Subsystem. 
This  level  decomposes  object  tasks  into  appropriate 
sequences  of  elementary  moves  defined  in  terms  of 
elementary  vehicle  manoeuvres  or  subsystem 
motions.  An  object  task  is  one  which  must  be  carried 
out  by  a  single  sub-system  or  on  a  single  object.  It 
also  coordinates  the’  start  and  stop  scheduling  of 
elementary  moves  and  the  scheduling  of  any 
coordinated  sub-svstem  activity  [Sample  rate  1Hz, 
replan  time  Is],  Level  5  -  Cooperating 

Robots/Service  Bay.  This  level  decomposes  complex 
tasks  involving  operations  on  a  number  of  objects 
into  tasks  carried  out  on  single  objects  [Sample 
0.1Hz,  replan  time  10s] .  Level  6  -  Mission/Groups 
of  Robots.  This  level  decomposes  an  overall  vehicle 
mission  plan  into  action  commands.  The  mission 
plans  would  usually  be  generated  by  humans  off-line 
and  specified  in  terms  of  mission’ goals,  priorities, 
requirements,  constraints  and  overall  mission 
timescales.  [Sample  rate  0.01Hz,  replan  time 
1.7minj. 

One  area  of  particular  importance  is  safety  and,  in 
order  to  prevent  the  vehicle  performing  undesirable 
actions  or  entering  undesirable  states,  a  separate 
safety  system  using  redundant  sensors  and  a 
completely  separate  data  base  may  be  necessary. 

NASREM  aims  to  define  a  set  of  guidelines  which 
can  evolve  into  applicable  standards  for  the  design 
and  implementation  of  a  wide  range  of  autonomous, 
semi-autonomous  and  tcleopcrated  systems, 
subsystems  and  components. 

Behavioural  Architectures  In  this  architecture  the 
control  task  is  broken  down  into  a  hierarchy  of 
behaviours  or  "competences",  each  of  which 
selectively  assists  or  assumes  (subsumes)  the  control 
functions  of  lower  behaviours.  Each  control  level  is 
built  incrementally  on  top  of  the  last  tested  and 
debugged  control  structure.  Strictly  speaking,  the 
layers  are  built  from  modules  that  are  finite  state 
machines  which  can  communicate  with  each  other 
via  simple  protocol,  narrow  bandwidth  links. 
Modules  can  inhibit  the  output  of  other  modules  and 
suppress  input  to  a  module  whilst  substituting  its 
own  data  in  place  of  the  suppressed  input.  This 
mechanism  enables  one  layer  of  modules  to  add 
behaviours  and  extend  the  behaviour  of  the  lower 
levels  so  that  the  total  set  of  behaviours  is  greater 
than  before.  Low  level  high  priority  safety 
behaviours  are  simple  to  ensure  rapid  response. 
More  complex,  less  critical  mission  specific 
behaviours,  such  as  map  building  or  scene 
reconstruction,  are  layered  over  the  safety 
behaviours.  The  significant  point  is  that  low  level 
survival  behaviours  do  not  have  to  wait  or  interact 
with  more  complex  behaviours.  Contention  between 
multiple  parallel  behaviours  for  control  of  an 
actuator  is  resolved  by  hierarchically  layering  the 
behaviour.  In  practice,  the  capacity  of  individual 
modules  is  soon  overloaded  and  considerable  effort  is 
necessary  to  prioritise  supposedly  parallel  finite  state 
machines.  In  order  to  allow  easy  and  rapid 
reconfiguration  of  the  control  structure,  both  during 
and  in  between  missions,  an  augmentation  of  the 
architecture  is  necessary.  By  applying  state 


configured  control  the  structure  is  simplified  and  the 
number  of  behaviours  active  at  any  time  is  reduced. 
Initially,  a  state  transition  diagram  related  to  the 
desired  mission  is  constructed  indicating  the 
behaviours  which  need  to  be  operable  during  the 
various  stages  of  the  mission.  A  layered  control 
structure  associated  with  each  of  the  states  in  the 
state  transmission  diagram  is  then  generated. 

Both  functional  and  behavioural  control 
decompositions  have  their  advantages  and 
disadvantages.  The  behavioural  decomposition  and 
state  transition  layer  are  essential  in  the  design  of  the 
control  strategy,  whereas  the  functional  or 
hierarchical  decomposition,  which  is  required  at 
every  behavioural  level,  is  important  in  the  physical 
realisation. 

Beyond  this  basic  architectural  philosophy  are  the 
longer  term  goals  of  achieving  interoperability, 
extensibility  (to  new  components)  and  scaleability  (to 
increasingly  complex  systems).  Work  is  currently 
aimed  at  an  Open  Command  and  Control  Reference 
Model  (Ref.  10)  and  is  based  on  the  NASREM 
functional  model.  It  concentrates  on  the  organisation 
of  a  distributed  intelligent  sensing  system  allowing 
the  hardware  and  software  to  be  treated  separately 
and  flexibly.  With  the  emphasis  on  modularity  and 
productivity  of  hardware  and  software,  the 
opportunity  for  much  greater  commonality  of 
thinking  and  cooperation  becomes  apparent,  and  will 
hopefully  provide  the  impetus  for  the  generation  of 
standards  and  reference  protocols  in  this  area. 

12.2  Control  Techniques 

Having  set  the  scene  to  give  an  overview  of  the  total 
system  navigation  task,  it  is  helpful  to  focus  on  the 
very  practical  problem  of  controlling  the  vehicle 
platform  itself.  The  two  basic  configurations  are 
wheeled  and  tracked.  The  fundamental  difference 
arises  in  the  method  of  steering  in  the  lateral 
direction.  The  wheeled  vehicle  achieves  this  by 
articulating  the  front  wheels,  whilst  in  the  tracked 
arrangement,  a  skid-steer  procedure  is  typically 
adopted  which  involves  differentially  powering  or 
braking  the  respective  tracks.  For  the  purpose  of  this 
discussion,  we  shall  concentrate  on  the  wheeled 
vehicle  concept  taking  as  an  example  the  ROVA 
vehicle  (Ref.  11).  This  is  a  fourwheeled  Camper 
Van  developed  as  an  autonomous  road  following  test 
bed  by  the  Defence  Research  Agency  in  UK.  The 
3.5  ton  vehicle  is  equipped  with  an  automatic 
gearbox  and  is  modified  for  computer  control  by  the 
addition  of  servo  actuators  for  the  steering, 
accelerator  and  service  brakes.  Television  cameras 
for  vehicle  guidance  are  mounted  on  a  steerable  head 
and  view  tne  scene  ahead  through  a  windscreen 
located  above  the  normal  driving  position.  The 
control  approach  is  that  described  by  Dickmanns 
(Ref.  12)  which  uses  model-based  control  to 
recursively  combine  prior  knowledge  (in  the  form  of 
dynamic  models)  and  current  measurements,  to 
estimate  the  motion  state  of  the  vehicle  and  states 
describing  the  road  geometry  which  act  as  a 
constraint  on  the  vehicle  motion.  This  method  is 
computationally  efficient,  provides  an  established 
treatment  of  measurement  and  system  noise,  allows 
the  selection  of  well  conditioned  measurements  (and 
conversely  the  rejection  of  outlying  measurements) 
and  provides  system  state  information  as  a  function 
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of  time.  The  scheme  may  be  expected  to  work 
satisfactorily  when  the  system  (i.e.  vehicle  and 
environment)  is  sufficiently  well  structured  to  be 
described  adequately  by  relatively  simple  models. 
We  can  assume  that  "the  vehicle  is  relatively  invariant 
and  unchanging  with  time,  whereas  the  environment 
is  likely  to  present,  even  in  a  road  scenario,  a  wide 
range  of  potential  road  boundaries,  junctions, 
obstacles,  traffic  etc.  which  will  tend  to  complicate 
the  road  scene  model.  Fig.  12  illustrates  the  control 
scheme  used.  For  correct  driving,  the  vehicle 
control  system  must  maintain  the  television  image  of 
the  road"  boundary  at  a  specified  position  in  the 
image.  In  the  basic  concept  the  lateral  location  of  the 
nearside  road  boundary  is  tracked  by  two  windows  in 
the  television  image  or  the  road  ahead  of  the  vehicle. 
Measurements  from  the  near  windows  are  used  by  a 
model  based  steering  controller,  which  incorporates  a 
Kalman  state  estimator,  to  derive  a  steering  demand, 
Uc,  which  maintains  the  vehicle  motion  (position  and 
heading)  within  safe  limits  on  the  road. 
Measurements  from  the  more  distant  windows  are 
used  to  update  a  Kalman  state  estimator  to  provide  a 
measure  of  road  curvature.  For  adequate  control 
bandwidth,  the  road  boundary  measurements  are 
made  at  half  frame  rate  giving  a  sampling  period  of 
T=80ms.  The  road  curvature  estimates  may  be  used 
for  vehicle  speed  control.  An  additional  anticipatory' 
control  block  ensures  that  the  optimum  vehicle 
motion  can  be  predicted  when  the  vehicle  moves  into 
a  curve.  The  general  form  of  the  steering  controller 
is  shown  in  Fig.  13.  If  it  is  assumed  that  the  vehicle 
(plant)  is  moving  on  a  plane  straight  road,  then, 
applying  small  angle  approximations,  the  open-loop 
vehicle  motion  at  a  given  speed  may  be  modelled  by 
5  linear  equations  in  the  state  vector  xT  =  (\|/,p,0,p,8) 
where, 

w  =  vehicle  yaw  rate 
p  =  sideslip  angle  at  the  centre  of  mass 
0  =  vehicle  body  heading  with  respect  to 
the  road 

p  =  lateral  displacement  of  the  vehicle  on 
the  road 

8  =  front  wheel  steer  angle 

In  standard  discrete  time  the  open-loop  motion  is 
modelled  by: 

x(k  +  1)  =  cDx(k)  +  Tu(k)  +  Tiw  (k)...  (1) 

where, 

u  =  control  input 
w  =  plant  noise  disturbance  vector 
<J>  =  plant  transition  matrix  representing  the 
plant  dynamics 
T  =  input  distribution  matrix 
T  i  =  noise  distribution  matrix 
k  =  time  in  units  of  the  sampling  period  T 

In  this  particular  representation  the  output  from  the 
plant  y,  is  the  measured  location  of  the  road 
boundary  in  the  image  plane.  This  is  obtained  bv 
forward  perspective  projection  and,  making  small 
angle  approximations,  is  linearly  related  to  the  states, 
thus 

y  =  Hx  +  v .  (2) 


where  v  is  the  measurement  noise  and  H  the 
measurement  matrix  which  depends  on  the  camera 
parameters  and  the  measurement  geometry. 

All  5  states  of  the  vehicle  are  estimated  from  this 
single  measurement  y  over  time,  using  a  standard 
Kalman  filter; 

x(k+l|  k)  =  <Px(k  |  k)  +  Fu(k) 

ye  —  Hx(k+l|  k)  .  (3) 

x(k  +  1 1  k  +1)  =  x(k  +  1 1  k)  +  L(y-ye) 

where  L  is  the  gain  of  the  filter  and  x(k+l|  k)  is  the 
state  estimate  at  time  (k+1)  based  on  measurements 
up  to  and  including  time  k.  ye  is  the  estimated  value 
of  y  at  time  (k+1).  L  depends  on  the  relative 
magnitudes  of  the  plant  and  the  measurement  noise. 
The  vehicle  compensatory  steering  demand  uc  is 
computed  by  state  feedback 

uc=  -Kx .  (4) 

where  the  control  gain  matrix  K  is  chosen  to  achieve 
a  satisfactory'  closed-loop  time  response. 

The  open-loop  vehicle  dynamics  are  speed  dependent 
so  that  in  equations  (1)  -  (4)  the  plant  transition 
matrix,  control  and  estimator  gain  matrices  are  also 
speed  dependent.  In  ROVA,  these  matrices  were 
precomputed  for  a  range  of  speeds  in  a  design  stage 
carried  out  off  the  vehicle,  and  stored  in  Took  up 
tables  which  were  addressed  according  to  speed  when 
the  vehicle  was  under  automatic  control. 

Control  and  Estimator  Gains  The  s-plane  locus  of 
the  poles  and  zeros  of  the  open-loop  system  are 
dependent  on  the  vehicle  speed.  There  were  3  fixed 
poles  at  the  origin,  corresponding  to  3  integrations 
which  arise  in  the  vehicles  equation  of  motion,  two 
fast  poles  (which  decay  rapidly  with  time)  on  the 
negative  real  axis  and  a  real  zero  relatively  close  to 
the  origin.  Suitable  values  of  K  were  prc-computed 
off-line  using  both  pole  placement  and  by 
optimisation  of  a  cost  function.  In  the  optimal 
approach,  the  cost  function  shown  in  eqn  (5)  was 
minimised  subject  to  cqn.(l).  Thus  I,  a  scalar 
measure  of  the  performance  of  the  control  system,  is 
given  by:  n 

I=S[xkTQi  xk  +  UkTQ2Uk]a2k .  (5) 

k-0 

The  parameter  a  allows  bounds  to  be  placed  on  the 
settling  time  of  the  closed  loop  system.  The  settling 
time,  and  the  weight  matrices  Qi  "and  Q2  were  chosen 
to  achieve  a  balance  between  dynamic  response  and 
the  control  effort  required.  Qi  and  Q2  allow  varying 
degrees  of  weight  to  be  placed  on  the  contribution  of 
the  states  x  and  the  magnitude  of  the  control  signal  u. 
For  a  computed  step  function  response  to  a  demand 
for  a  new  lateral  location  on  the  road,  an  acceptable 
rise  time  and  overshoot  were  obtained  over  a  range 
of  vehicle  speeds.  The  Kalman  gain  matrix  L  (steady 
state)  for  the  vehicle  state  estimator  was  computed 
using  assumed  values  for  the  measurement  and  plant 
noise  contributions. 

Road  Curvature  Estimation  Road  curvature  is 
estimated  using  road  boundary  measurements  made 
further  ahead  of  the  vehicle  in  conjunction  with  a 
geometric  road  model.  A  Kalman  filter  is  again  used 
to  obtain  an  optimal  estimate.  Account  must  be 
taken  of  the  vehicle  motion  state  as  this  contributes 
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to  the  measured  location  of  the  road  boundary  in  the 
tracking  window.  Similarly  when  the  vehicle  is 
moving  on  a  curved  road,  it  is  also  necessary  to 
include  curvature  terms  in  the  vehicle's  equations  of 


are  passed  to  the  vehicle  speed  controller,  to  regulate 
the  vehicle  speed  according  to  the  road  curvature. 

Speed  Control  Closed  loop  speed  control  is 
achieved  by  a  PI  controller  whose  parameters  were 
estimated  experimentally.  In  operation,  the  vehicle 
is  commanded  from  the  system  console  to  accelerate 
at  a  given  rate  to  a  final  desired  speed,  subject  to  a 
maximum  value  determined  by  the  local  road 
curvature. 

Control  System  Simulation  To  test  the  control 
system  design  and  software  prior  to  control  of  the 
physical  vehicle  a  real-time  closed  loop  simulation 
was  developed.  The  real  world  road-view  was 
replaced  by  a  computer  graphically  generated  view 
of  the  edges  of  the  road,  and  the  vehicle  itself  was 
represented  by  the  dynamic  model  given  in  cqn.(l). 
Each  road  edge  was  obtained  by  integration  of  the 
Frenet-Serrct  equations  for  a  plane  curve.  The 
control  cycle  time  used  was  80ms  as  in  the  real 
system.  Each  near  view  of  the  road  was  obtained  by 
integration  of  the  lateral  (eqn.l)  and  longitudinal 
equations  of  motion  of  the  vehicle  with  a  time  step  of 
10ms,  followed  by  projection  of  the  road  view  into 
the  image  plane.  The  road  location  was  measured  in 
the  image  plane  with  new  road  position  information 
injected  at  80ms  intervals,  and  a  steering  demand 
signal  was  input  to  the  vehicle  model  to  maintain  the 
correct  motion  of  the  vehicle  with  respect  to  the  road. 
The  simulation  was  carried  out  using  a  computer 
generated  circular  road  of  varying  curvature  and 
road  width,  and  establishing  the  vehicle  performance 
over  a  range  of  vehicle  speeds  and  distance  from  the 
road  edge  for  different  values  of  K  and  L.  By 
carrying  out  sensitivity  checks  in  this  way,  the 
number  of  vehicle  parameters  which  had  to  be 
measured  experimentally  was  minimised.  The 
simulation  was  executed  directly  on  the  vehicle's 
transputer  machine  to  ensure  direct  applicability  and 
to  reduce  the  need  to  rewrite  code.  16  transputers 
were  used  in  the  overall  system  simulation,  6  being 
dedicated  to  the  vehicle  dynamics  and  road  generator, 
which  were  not  required  when  the  vehicle  was  driven 
live. 

Control  System  Design  Procedure  The  following 
summarises  the  sequence  of  operations  which  were 
found  necessary  in  the  development  of  the  ROVA 
control  system. 

1.  Model  the  Plant.  Derive  the  equations  of 
motion  in  terms  of  appropriate  state  variables. 

2.  Discrete  Time  Plant  Matrices.  Choose  the 
sample  time  of  the  digital  control  system  and 
compute  the  matrices  ®,  F  and  H  from  equations  of 
motion. 

3.  Pole  Placement  Routines  to  compute 
Feedback  Gain  K  and  Estimator  Gain  L.  To  ensure 
stability  and  achieve  desired  time  response  under  the 
range  of  chosen  conditions,  eg.  speed,  Kalman 
estimation  can  be  used  to  determine  L  depending  on 


the  relative  influence  of  plant  and  measurement 


4.  Reference  Demand.  Allows  choice  of  the 
control  system  set  point,  e.g.  lateral  distance  from  the 
road  boundary. 

5.  Time  Response  Computation  and  Display. 
Examine  the  time  response  of  the  system  in 
simulation  (non-real  time)  to  changes  in  the  set  point. 
If  unsatisfactory,  repeat  procedure  from  Step  3  until 
the  performance  is  acceptable.  Investigate  sensitivity 
to  changes  in  ®,  T  and  H. 

6.  Control  Parameter  Data  File.  Collect 
together  all  speed  dependent  elements  of  ®,  F,  H,  K, 
L  and  the  reference  demand  into  a  look-up  table  for 
use  by  the  vehicle  controller  for  autonomous  driving. 

7.  Real  Time  Simulation.  Run  the  system  over 
a  test  course  to  confirm  software  integrity. 

8.  Experimental  Trials.  Confirm  expected 
performance  in  autonomous  driving  with  the  real 
vehicle  system  parameters  ®,  F  and  H. 

12.3  Application  of  Fuzzy  Logic  to  Control 

Classical  control  techniques,  like  the  model 
reference  approach,  result  in  precise  deterministic 
performance  at  the  expense  of  the  complex 
formulation  of  dynamic  models.  The  development  of 
fuzzy  logic  attempts  to  reverse  this  trend  by 
generating  pseudo  qualitative  methods  which  rely  on 
finite  and  generally  small  numbers  of  outcome  values 
formed  as  a  result  of  combinations  of  limited 
quantised  input  parameters.  Take  as  a  simple 
example  a  vehicle  travelling  along  an  obstacle  free 
corridor  (Ref.  13).  Given  the  object  is  to  centre  the 
vehicle  motion  along  the  centre  of  an  initially  linear 
corridor,  a  rule  structure  has  to  be  set  up  to  define 
two  condition  variables,  the  heading  relative  to  the 
corridor,  and  the  distance  from  the  corridor  centre 
line.  The  lateral  pilot  has  to  produce  an  output 
which  is  the  desired  vehicle  path  curvature.  Thus  if 
the  heading  is  A  and  the  offset  distance  is  Bi  then 
the  desireef  curvature  is  G.  The  fuzzy  set  definitions 
for  this  corridor  regulator  are  illustrated  in  Fig.  14. 
The  input  variables  are  classified  as  being  large, 
medium  or  small,  positive  or  negative,  or  zero.  A 
triangular  membership  distribution  p  using  intuitive 
limits  on  the  values  is  chosen  with  a  non-linear 
distribution  around  the  origin  to  provide  increased 
sensitivity.  Thus  seven  fuzzy  sets  are  used  on  the 
two  condition  variables,  leading  to  a  possible  49 
rules.  These  are  collapsed  to  11  possible  outcomes 
using  a  verbalisation  technique,  i.e.  by  describing  the 
actions  of  a  human  driver  in  this  rule  format.  By 
adding  the  measured  path  curvature  at  a  fixed 
distance  ahead  (the  look  ahead  distance)  to  the  output 
of  the  corridor  regulator,  path  guidance  can  be 
generated  for  small,  constant  curvature  corridors. 
Reducing  the  corridor  width  demands  a  tighter 
vehicle  response  and  correspondingly  higher 
maximum  path  curvature  demands.  Having 
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vehicle's  speed  along  the  path.  This  requires 
inclusion  or  the  vehicle's  dynamics  which  are  usually 
parameterised  by  speed  as  well  as  environmental 


Set  Point 


Fig. 15  The  Mamdani  self-organizing  controller 
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conditions  such  as  road  adhesion.  This  procedure 
provides,  therefore,  an  alternative  approach  to  the 
simulation  previously  described  for  the  model 
reference  system  applied  to  ROVA. 

So  far  we  have  described  a  static  fuzzy  logic 
controller  which  is  robust  and  adept  at  dealing  with 
complex  or  poorly  defined  dynamical  systems,  but  it 
offers  no  on-line  adaptation  to  reflect  changes  which 
may  occur  in  the  controlled  process  or  its  interaction 
with  the  environment.  The  adaptive  fuzzy  controller 
overcomes  these  problems  by  performing  on-line 
adaptation  of  the  controller  rule-base.  These 
adaptive  controllers  can,  if  sufficiently  excited, 
produce  a  complete  rule-base,  replacing  inadequate 
or  faulty  rules.  The  most  widely  used  technique  is 
the  self-organising  fuzzy  logic  control  introduced  by 
Mamdani  which,  as  with  any  learning  system, 
requires  a  performance  measu re/index  against  which 
the  achieved  response  can  be  evaluated.  This  can 
then  be  used  to  generate  a  feedback  signal  to  alter  the 
control  policy.  In  this  way,  a  self-organising 
controller  is  able  to  improve  its  capability  based 
purely  on  experiential  means,  with  little  or  no  a  priori 
knowledge  of  the  process  being  necessary.  The 
Mamdani  controller  is  illustrated  in  Fig.  15.  The 
systems  break  down  into  two  levels,  the  controller 
and  the  adaptation  level.  The  controller  level  is 
identical  to  that  used  in  the  static  fuzzy  controller, 
described  earlier,  with  the  nde-base  supplied  from 
the  adaptation  level.  The  adaptation  level  uses  a 
performance  table  to  drive  a  rule  modification 
process.  A  typical  performance  table  in  terms  of 
error  and  error  rate  is  shown  in  Fig.  16.  A  central 
band  consists  of  desired  combinations  of  error  and 
error  rates  which  require  no  further  adaptation. 
Regions  away  from  this  desired  band  generate  an 
adaptation  signal  which  determines  the  direction  of 
the  rule  adaptation.  The  rule  modification  process 
involves  identifying  the  rule  which  causes  poor 
performance  (usually  by  selecting  the  rule  which  was 
used  at  a  fixed  time  interval  in  the  past)  and  applying 
the  modification  required  by  the  adaptation  signal. 

The  following  summarises  the  attributes  of  adaptive 
fuzzy  logic  controllers: 

1.  the  ability  to  control  systems  with  little  a  priori 
plant  knowledge. 

2.  fast  adaptation  to  large  unmeasured  plant 
parameter  changes. 

3.  effective  operation  in  the  presence  of  a  mismatch 
between  plant  order  and  controller  rule  structure 
dimension. 

4.  control  of  plants  with  stationary  non-linearities. 

5 .  good  noise  and  disturbance  rejection  capability. 

6.  probable  convergence  for  fuzzy  plant  modelling 
(under  strict  technical  conditions). 

Unfortunately'  the  memory  requirements  for  fuzzy 
logic  controllers  grow  exponentially  with  the 
dimension  of  the  system  variables  used  m  the  control 
rule-base.  However,  it  is  possible  to  structure  rules 
for  a  static  fuzzy  controller  in  a  hierarchical  or 
nested  fashion  to  reduce  the  number  of  rules  for 
complete  cover. 


Fig. 16  A  typical  performance  table  for 
a  self-organizing  controller. 


12.4  Artificial  Neural  Networks  for  Control 

Artificial  neural  systems  can  be  designed  to  imitate 
rules  of  behaviour  that  permit  them  to  function 
autonomously  in  a  work  space,  after  they  have  been 
trained  to  respond  correctly  to  a  set  of  situations 
which  spans  the  range  of  those  likely  to  be 
encountered.  The  training  can  be  provided  manually, 
either  under  the  direct  supervision  of  a  system 
trainer,  or  indirectly  using  a  background  mode 
where  the  network  assimilates  training  data  as  the 
expert  performs  his  day  to  day  tasks.  Artificial 
neural  systems  are  networks  which  consist  of  a 
number  of  processing  elements  interconnected  in  a 
weighted,  user-specified  fashion,  the  interconnection 
weights  acting  as  memory'  for  the  system.  Each 
processing  element  calculates  an  output  value  based 
on  the  weighted  sum  of  its  inputs.  In  addition,  the 
input  data  are  correlated  with  the  output  or  desired 
output  (specified  by  an  instructor)  in  a  training 
exercise  that  is  used  to  adjust  the  interconnection 
weights.  In  this  way,  the  network  learns  patterns  or 
imitates  rules  of  behaviour  and  decision  making.  The 
Multilayer  Perceptron  (MLP)  is  one  variant  of  an 
artificial  neural  network  which  has  been  found 
suitable  for  many  applications  (Fig.  17).  The  input 
and  output  layers  of  nodes  are  connected  by  one  or 
more  intermediate  or  "hidden"  layers,  and  each 
interconnecting  link  is  associated  with  a  weighting 
value.  One  of  the  early  examples  of  an  artificial 
neural  network  (ANN)  was  one  designed  to  drive  a 
robot  vehicle  down  a  simulated  highway  in  traffic 
(Ref.  14).  The  fully  developed  highway  simulation 
consisted  of  a  two  lane  highway  having  varying 
straight  and  curved  segments.  Several  pace  cars 
moved  at  random  speeds  near  the  robot  vehicle.  The 
network  was  given  the  tasks  of  tracking  the  road, 
negotiating  curves,  maintaining  safe  distances  from 
the  pace  cars,  and  changing  lanes  w'hen  appropriate. 
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Fig. 17  Structure  of  Multilayer  Perceptron 
used  for  Texture  Classification. 

Instead  of  a  single  multi-layer  structure,  the  network 
was  composed  of  two  blocks:  one  controlling  the 
steering  and  the  other  regulating  the  speed  and 
deciding  when  the  vehicle  should  change  lanes 
(Fig.  18).  The  first  block  received  information  about 
the  position  and  speed  of  the  robot  vehicle  relative  to 
other  cars  in  its  vicinity.  Its  output  was  used  to 
determine  the  vehicle's  speed  and  whether  the  robot 
should  change  lanes.  The  passing  signal  was 
converted  to  a  lane  assignment  based  on  the  car's 
current  lane  position.  The  second  block  received  the 
lane  assignment  and  data  pertinent  to  the  position 
and  orientation  of  the  vehicle  with  respect  to  the 
road.  The  output  was  used  to  determine  the  steering 
angle  of  the  robot  car.  The  two  blocks  were  trained 
separately.  This  was  found  to  be  very  effective 
because  each  block  had  a  restricted,  well-defined  set 
of  tasks,  and  the  trainer  could  concentrate 
specifically  on  those  functions  without  being 
concerned  that  other  aspects  of  the  network 
behaviour  might  be  deteriorating.  The  system  was 
trained  from  bottom  up,  first  teaching  the  network  to 
stay  on  the  road,  negotiate  curves,  change  lanes,  and 
how  to  return  if  the  vehicle  strayed  off  the  highway! 
Block  2,  responsible  for  steering,  learned  these  skills 
in  a  few  minutes  using  the  master/apprentice  or 
supervised  training  mode  in  which  the  trainer 
actuates  the  vehicle  and  the  network  takes  the 
trainer's  actions  as  the  desired  system  responses  and 
correlates  these  with  the  input.  The  ANN  tended  to 
steer  more  slowly  than  a  human  but  further  training 
progressively  improved  its  responsiveness.  After 
Block  2  was  trained,  steering  control  was  handed  to 
the  ANN  in  order  to  concentrate  on  teaching  the 
network  to  change  lanes  and  adjust  speed.  Speed 
control  in  this  case  was  a  continuous  variable  and 
was  best  taught  using  mastcr/apprentice  training.  On 
the  other  hand,  the  binary  decision  to  change  lanes 
was  best  taught  by  coaching.  In  this  case  tne  ANN 
remained  in  direct  control  of  the  robot  vehicle,  but 
the  human  trainer  instructed  it  when  and  when  not  to 
change  lanes.  The  network  quickly  correlated  its 
environmental  inputs  with  the  teacher's  instructions 


and  after  only  ten  minutes  of  training,  the  network 
became  adept  at  changing  lanes  and  weaving  through 
traffic.  It  was  found  that  the  network  readily  adapted 
to  the  behavioural  pattern  of  its  trainer.  A 
conservative  trainer  generated  a  network  that  hardly 
ever  passed  other  cars,  whilst  an  aggressive  trainer 
produced  a  network  that  drove  recklessly  and  tended 
to  cut  off  other  cars.  This  example  demonstrated  the 
ability  of  an  artificial  neural  system  to  operate 
satisfactorily  in  an  environment  where  the  rule  set 
governing  an  expert's  decisions  is  difficult  to 
formulate.  Also  imitations  of  expert  behaviour 
tended  to  emerge  as  a  natural  consequence  of  their 
training. 

Neural  Network  for  Autonomous  Navigation.  We 

have  seen  that  an  Artificial  Neural  Network  (ANN) 
can  be  used  to  control  a  vehicle  successfully  when 
presented  with  a  limited  range  of  discrete  input 
parameters.  This  approach  can  be  extended  by 
presenting  the  ANN  with  a  pre-processed  image 
derived  from  a  video  sensor  viewing  the  scene  ahead 
of  the  vehicle  and,  with  suitable  training, 
conditioning  the  output  to  provide  steering 
commands  for  the  vehicle  so  that  it  drives  safely 
along  the  desired  route.  A  system  known  as 
ALVINN  (Autonomous  Land  Vehicle  in  a  Neural 
Network)  has  been  designed  at  Carnegie  Mellon 
University  to  drive  the  Navlab,  their  autonomous 
navigation  test  vehicle  in  this  way.  (Ref.29).  The 
ALVINN  architecture  is  of  the  Multilayer  Perceptron 
form  (Fig.  19)  with  a  single  hidden  layer  back- 
propagation  (supervised)  network.  The  input  layer 
of  tne  network  consists  of  a  "retina"  having  30  x  32 
units  on  to  which  the  video  camera  image  is  projected 
after  preprocessing.  The  output  array  is  a  vector, 
typically  30  elements,  in  which  each  element 
represents  the  strength  of  votes  for  a  particular 
steering  direction.  Tne  middle  layer  consists  of  four 
hidden  units.  Each  hidden  unit  is  connected  to  all 
960  inputs  and  to  all  30  outputs.  The  system 
therefore  contains  3960  total  weights,  3840 
connecting  the  input  to  the  hidden  units  and  120 
connecting  the  hidden  units  to  the  output  units. 
These  weights  describe  the  representation  of  the 
road,  and  consequently  the  processing  the  neural  net 
has  to  perform  to  generate  the  steering  commands 
from  the  road  images.  ALVINN  is  trained  by 
watching  a  human  drive,  using  the  video  images  as 
model  input  and  human  steering  direction  as  desired 
output.  The  training  process  sets  the  weights  for  the 
interconnections.  In  a  few  minutes  of  training, 
ALVINN  learnt  how  to  drive  on  a  particular  type  of 
road. 

The  colour  input  images  are  pre-processed  to 
produce  a  single-band  output  image.  Each  pixel  in 
the  image  is  normalised  to  have  a  value  between  0 
and  1 .  The  normalised  value  is  given  by: 


aB  1 

+ 

(l-a)B 

_  255  j 

L  R+G+B  J 

where  R,  G  and  B  are  the  raw  red,  green  and  blue 
values  for  a  particular  pixel;  a  is  a  weighting  factor. 
This  normalisation  provides  some  tolerance  to 
lighting  variation.  While  the  R,  G  and  B  values  will 
change  from  sunlit  to  shadowed  areas,  the  ratios  of 
the  values  will  stay  approximately  the  same. 
Empirically,  the  blue  band  contains  the  most  useful 
contrast  for  road  following. 
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Fig. 18  The  two  blocks  of  the  driving  neural  network.  Heavy  arrows  indicate  total 

interconnectivity  between  layers.  PL  designates  the  traffic  lane  presently  occupied 
by  the  robot  vehicle,  OL  refers  to  the  other  lane,  curvature  refers  to  road,  the  lane 
number  is  either  0  or  1 ,  relative  orientation  and  lateral  distance  refer  to  the  robot 
car's  direction  and  position  relative  to  the  road's  direction  and  centre  line,  respectively. 


Of  the  video  images,  digitised  to  480  rows  by  512 
columns,  some  10%  of  the  pixels  are  sampled  and 
averaged  to  feed  the  input  layer.  This  preserves 
some  of  the  essential  variability  of  the  underlying 
image,  whilst  providing  adequate  smoothing  in 
reducing  the  image  size.  The  desired  output  is  a 
steering  direction.  The  actual  representation  used  by 
ALVINN  is  a  vector  of  outputs.  Each  element  of  the 
output  vector  collects  votes  from  the  hidden  units  for 
or  against  the  steering  direction  which  it  represents. 
By  fitting  a  Gaussian  to  the  area  around  the  peak 
activation  (most  positive  votes)  or  by  finding  the 
centre  of  mass  of  votes  in  the  neighbourhood  of  the 
peak,  a  continuously  varying,  well  behaved  steering 
output  direction  was  obtained. 

Back  Propagation  Learning.  The  training  phase 
generated  the  weights  connecting  input  to  hidden 
units,  and  hidden  units  to  die  output.  Back 
propagation  learning  was  used  in  which  the  weights 
were  initially  set  to  random  values.  At  each  step  of 
training,  the  system  was  run  forward,  from  input 
pixels  to  output  steering  directions.  The  reported 
output  was  compared  with  the  desired  output.  Errors 
in  the  output  were  used  to  adjust  the  weights  that  led 
to  those  output  units.  The  process  was  run 
recursively  at  the  hidden  units,  propagating  the  errors 
(and  the  corrections)  back  to  the  inputs.  Back 
propagation  was  continued  until  the  errors  were 
acceptably  low.  Several  subtleties  arose  in 


generating  the  training  images  used  in  learning.  If 
the  onlv  images  used  came  from  a  human  performing 
perfectly  at  driving,  ALVINN  would  never  learn  how 
to  recover  from  minor  steering  errors.  This  was 
solved  by  creating  derived  training  images  from  the 
actual  images.  Starting  with  a  given  input  image  and 
desired  steering  direction,  the  image  was  shifted  or 
rotated  slightly  to  create  the  image  that  the  vehicle 
would  see  if  it  were  slightly  off  the  desired  path.  The 
desired  steering  angle  was  corrected  correspondingly. 
This  gave  ALVINN  a  much  broader  training  set 
aimed  at  encompassing  the  range  of  fikely 
experience.  Another  subtlety  involved  the  need  to 
avoid  bias:  if  the  training  used  images  only  from  left 
turns,  ALVINN  would  learn  that  always  turning  left 
minimised  output  errors.  The  results  obtained  with 
ALVINN  have  been  impressive,  the  vehicle  being 
driven  recently  for  145km  at  speeds  up  to  113kph. 
However,  the  performance  is  still  inadequate  for 
realistic  applications,  and  sufficient  levels  of 

enerality,  reliability'  and  self  diagnosis  have  not 
een  achieved  which  would  allow  the  vehicle  to  run 
unattended  in  a  wide  variety  of  situations.  The 
approach  adopted  is  very'  ambitious  with  the 
relatively  simple  neural  network  being  tasked  with 
both  image  understanding  and  constructive  controller 
response.  In  fact,  current  research  is  now  being 
directed  at  understanding  more  fullv  the  behaviour  of 
the  inherent  ALVINN  system.  Tor  instance,  how 
much  does  the  design  of  the  neural  network 
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Fig-19  Neural  network  architecture 
for  autonomous  driving. 

contribute  to  the  performance  by  comparison  with 
the  training  set  generation  or  the  output 
representation? 

Haying  described  both  fuzzy  logic  control  and 
artificial  neural  networks,  we  can  see  that  they  have 
some  striking  similarities.  They  both  attempt  to  deal 
with  real-time  adaptive  operation  of  complex  linear 
and  non-linear  systems.  In  particular,  a  class  of 
neural  systems  known  as  Associative  Memory 
Neural  Networks  resembles  fuzzy  models  under 
certain  conditions.  These  have  the  advantage  over 
the  more  popular  Multilayer  Perceptron  (MLP)  that 
they  enable  a  satisfactory  system  performance  to  be 
obtained  over  widely  differing  sets  of  situations. 

12.5  Road  Detection  Systems 

We  have  seen  how  the  UGV  can  be  controlled  in  a 
predictable  way  when  provided  with  reliable  steering 
input  information.  This  information  has  to  be 
derived  from  some  sensing  of  the  local  surroundings. 
In  the  case  of  cross  country  navigation,  this  becomes 
a  task  of  detecting  obstacles  and  discriminating 
gradients  that  the  vehicle  can  negotiate.  But,  as  with 
manned  vehicles,  the  majority  of  driving  will  be  on 
prepared  roads  or  at  least  on  unprepared  surfaces 
where  there  are  some  delineating  features  which 
determine  the  trackway.  The  problem  of  autonomous 
navigation  can  only  'be  solved,  therefore,  if  a 
reliable  system  of  automatic  road  detection  can  be 
developed. 

In  principle,  there  are  two  stages  to  the  problem. 
Firstly,  there  is  the  recognition  of  the  road  in 
question  from  some  off  road  viewpoint,  which  will 
allow'  subsequent  access  to  the  road  and,  secondly, 
when  access  has  been  gained,  there  is  the  need 
continuously  to  identify  the  road  surface  to  enable 


the  vehicle  to  stay  on  the  road.  The  first  stage  is  an 
example  of  the  general  problem  of  scene  or  image 
understanding,  whilst  the  second  is  somewhat  easier, 
in  that  image  comparisons  of  more  distant  view's  can 
be  made  with  the  immediate  foreground,  on  the 
reasonable  assumption  that  the  road  surface 
appearance  will  not  change  dramatically  in  the 
vicinity  of  the  vehicle.  This  second  stage  must  also 
allow’  continuous  monitoring  of  the  road  boundaries 
and  the  differentiation  between  intersecting  roads  at 
junctions.  Here  we  discuss  some  of  the  methods  for 
discriminating  roads  in  the  global  surroundings.  We 
have  already  described  systems  which  do  not 
explicitly  identify  roads  from  an  analysis  of  their 
physical  properties.  ALV1NN,  for  instance,  is 
trained  on  image  data  derived  from  a  range  of  road 
scenes.  We  shall  concentrate  now  on  explicit 
techniques  for  road  detection  and  this  implies 
identifying  those  features  of  a  road  embedded  scene 
which  allow  the  road  to  be  separated  from  its 
surroundings.  Broadly  the  features  are  edge  or 
region  based. 

Edge  Techniques  The  edge  features  can  be 
enhanced  by  the  application  of  a  convolution 
operator,  e.g.  Sobel  operator,  to  the  digitised  video 
image.  This  should  enable  the  road  boundaries  and 
road  markings  to  be  highlighted,  and  confirmation  of 
the  road  edges  obtained  by  comparison  with  a  stored 
model  of  a  representative  road.  The  system  adopted 
by  Dickmanns  (Ref.  12)  used  a  limited  number  of 
localised  edge  trackers  to  track  the  road  edges.  This 
allowed  rapid  computation  at  frame  rates,  and 
enabled  the  VaMoRs  vehicle  to  achieve  runs  at  up  to 
96  kph.  Whilst  the  technique  correctly  focused 
attention  on  the  road  edges  themselves,  and  gained 
some  stability  by  being  model-based,  the  trackers 
could  be  seduced  by  spurious  edges  arising  from 
shadows  and  changes  in  road  structure  such  as 
intersections.  A  more  sophisticated  approach 
(Ref.  16)  illustrated  the  processes  involved  in 
producing  a  more  robust  solution  to  the  problem. 
The  camera  image  is  transformed  into  a  plan  view’ 
image  which  has  the  advantage  that  edge  detection 
can  be  simplified  to  a  vertical  edge  detector.  There 
is,  then,  a  sequence  of  operations  which  involves 
searching  for  edge  segments,  linking  the  edge 
segments,  fitting  road  edge  candidates  mto  straight 
lines,  selecting  a  pair  of  straight  lines  (the  right 
boundary’  and  the  left),  extending  the  lines  with 
relevant  linked  segments,  to  overcome  the  problem  of 
interrupted  edges,  and  fitting  the  lines  with  two 
second-order  polynomials  to  obtain  the  road 
curvature.  Processing  this  sequence  of  operations 
was  accomplished  at  a  rate  of  6  frames/sec. 

Area  Techniques  Area  or  region  based  techniques 
rely  on  the  characteristics  of  the  road  surface  area 
being  distinguishable  from  the  roadside  verges. 
Texture,  gray  level,  and  colour  measures  can  be  used 
as  discriminants.  AH  suffer  from  the  inability  to  deal 
with  widely  variable  characteristics  of  real  road 
surfaces.  Gray  level  sensing,  for  instance,  depends 
on  accumulating  histogram  data  over  the  video  scene 
image,  and  setting  a  threshold  to  differentiate  the 
road  from  verge.  The  imaged  road  surface  intensity 
level  will  depend  on  surface  reflectivity',  the  sun 
direction,  the  degree  of  shadowing,  and  the  nature  of 
the  verge  surface,  all  of  which  will  cause  the 
optimum  threshold  for  road  discrimination  to  change. 
Colour  adds  an  extra  dimension  to  feature 
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measurement,  and  an  adaptive  colour  classification 
system  SCARF  (Supervised  classification  applied  to 
road  following)  has  been  developed  by  CMU 
(Ref.  17)  to  avoid  some  of  the  shortcomings  of  the 
monochrome  techniques.  SCARF  typically  uses  four 
colour  classes  to  describe  road  appearance,  and  four 
to  describe  off  road  objects.  In  addition,  the  multiple 
colour  classes  arc  represented  as  Gaussian 
distributions  in  full  red-green-blue  colour,  so  that  the 
probability  of  a  pixel  being  assigned  to  a  particular 
class  can  be  determined  instead  of  using  binary 
thresholds.  Multiple  classes  let  SCARF  represent 
the  different  colours  of  the  road  (such  as  asphalt, 
wet  patches,  shadowed  pavement,  and  leaves)  and  off 
roaa  objects  (such  as  trees,  sunlit  grass,  shaded 
grass,  and  leaves).  Classified  pixels  vote  for  all  road 
locations  that  would  contain  them  with  votes 
weighted  by  classification  confidence.  The  road  with 
the  most  votes  is  used  both  for  steering  and  for 
recalculating  the  colour  classes  using  nearest-mean 
clustering  to  collect  new  road  and  off-road  colour 
characteristics.  SCARFs  simple  model  of  road 
geometry  represents  roads  as  triangles  in  the  image. 
The  apex  is  constrained  to  lie  on  a  particular  row' 
(the  horizon)  and  the  base  has  a  fixed  width, 
dependent  on  the  road  width  and  the  camera 
calibration.  There  are  two  free  parameters:  the 
column  in  which  the  apex  appears  and  the  skew  of 
the  triangle.  While  this  simple  two-parameter  model 
does  not  represent  curves,  hills,  or  road  width 
variations,  it  does  approximate  the  road  shape  well 
enough  to  allow  reliable  driving.  It  is  especially 
effective  because  the  voting  procedure  uses  all 
pixels,  not  just  those  on  the  edges,  and  therefore  is 
relatively  insensitive  to  misclassifications. 
Furthermore,  the  simple  model  allows  for  fast  voting, 
and  highly  reduced  images  (typically  60  x  64  or  30  x 
32  pixels)  can  be  processed  at  high  sampling  rates 
(approximately  2  seconds  per  frame).  This  enables 
small  errors  in  road  representations  to  be  corrected 
before  the  vehicle  arrives  at  the  mistaken  locations. 
Even  drastic  illumination  changes  caused  by  clouds 
covering  the  sun  are  perceived  as  gradual  shifts  m 
road  appearance.  SCARF  has  driven  the  Navlab  on 
various  roads:  bicycle  paths,  dirt  roads,  gravel  roads 
and  suburban  streets.  The  system  has  been  extended 
by  including  in  the  road  model  the  admissibility  of 
intersections. 

The  road  recognition  task  is  also  suited  to  the 
application  of  an  artificial  neural  network  (Ref.  18). 
A  Multilayer  Perceptron  (Fig.  17)  has  been  designed 
to  operate  as  a  texture  classifier  capable  of 
distinguishing  between  three  possible  candidates, 
namely'  "road",  "verge"  and  1  sky".  The  texture 
measures  used  were  the  standard  deviation  of 
differences  in  intensities  AI  between  pairs  of  pixels  at 
1.  2,  4  and  8  pixel  separations  Ax,  which  can  be 
simply  and  rapidly  computed.  Fractal  textures  are 
characterised  by  a  linear  plot  of  log  AI  against 
log  Ax,  with  rougher  textures  having  a  steeper 
gradient.  There  is  sufficient  difference  between  the 
characteristic  for  the  3  candidate  classes  to  make 
classification  effective.  The  MLP,  with  8  texture 
measures  (4  horizontal  and  4  vertical)  as  input  and 
one  hidden  layer  with  2  nodes,  was  trained  on  a 
range  of  country  road  scenes  and  achieved 
successful  road  classification  on  scenes  taken  from 
different  geographical  areas.  Real  time  operation 
was  achieved  using  a  network  implemented  with 
transputers. 


12.6  Mission  Planning 

An  essential  part  of  the  tasking  of  an  autonomous 
UGV  is  to  define  the  route  to  the  final  goal  or 
destination.  This  may  be  via  a  network  of  roads  or 
tracks  or  across  country.  Typically  a  map  at  a  scale 
of  1 :50,000  can  be  used  to  create  a  network  of  nodes 
and  route  segments  from  w'hich  an  optimum  route 
can  be  planned.  This  optimum  route  may  be  chosen 
on  the  basis  of  minimum  distance,  minimum 
travelling  time,  or  any  other  "cost"  parameter.  This 
so  called  A*  algorithm  is  one  of  a  number  of  route 
optimisation  techniques  which  can  be  applied  either 
before  the  mission  commences  or  on-line  during  the 
course  of  the  mission.  The  A*  algorithm  takes  the 
"cost”  (or  optimising  parameter)  accumulated  in 
reaching  a  given  node  from  the  start  node,  together 
with  an  estimate  of  the  distance  remaining  to  the  goal 
node,  to  guide  the  search  process  for  selecting  the 
next  node  to  be  aimed  for.  Having  chosen  the  overall 
route  to  the  goal  destination,  various  key  waypoints, 
intersections  and  other  landmarks  can  be  selected  and 
used  as  the  necessary'  a  priori  information  for 
"updating"  the  vehicle  position  in  global  co-ordinates 
when  the  vehicle  sensing  system  identifies  them  en 
route.  A  route  planner  designed  to  support  a  wide 
variety  of  digital  terrain  databases  has  been 
developed  by  JPL  (Ref.  19).  This  route  planner 
generates  a  3D  composite  cost  surface  for  the  map 
area.  It  consists  of  a  rectangular  grid  of  nodes  with 
each  node  of  the  surface  corresponding  to  a  point  on 
the  ground  whose  "height"  (i.e.  cost)  mav  depend  on 
various  attributes.  These  may  include,  for  example, 
elevation  of  the  point,  steepness  of  gradients, 
presence  of  water,  marsh,  or  enemy  threat.  The 
relative  influence  of  these  factors  on  the  cost  is 
defined  by  the  human  mission  planner  The 
composite  cost  surface  is  constructed  for  the 
geographic  region  under  consideration,  and  the  route 
planner  attempts  automatically  to  find  the  minimum 
cost  path  between  start  and  end  points  using  the 
nodes  of  the  composite  cost  surface.  The  A 
algorithm  is  used  to  generate  the  minimum  cost  path. 
Particular  features  can  be  avoided  by  associating  a 
high  cost  with  those  nodes  where  the  feature  exists. 
In  general,  nodes  associated  with  roads  can  be 
expected  to  merit  low'  cost,  but  as  the  planner  is  free 
to  explore  routes  via  any  of  the  available  nodes  in  the 
composite  surface,  this  approach  will  allow  cross 
country  routes  to  be  proposed  if  these  prove  more 
favourable. 

12.7  Route  Planning 

We  refer  to  route  planning  as  the  local  path  planning 
w'hich  needs  to  be  carried  out  by  the  autonomous 
vehicle  in  its  attempt  to  execute  the  mission  plan.  It 
is  an  essential  part  of  the  sense-plan-drive  cycle 
which  governs  the  motion  of  the  UGV.  Three 
constraints  can  be  defined  which  are  important  for 
enabling  a  safe  trajectory  to  be  planned  in  order  to 
reach  the  goal,  namely  sensing,  environmental  and 
kinematic.  Firstly,  positions  have  to  be  selected 
where  the  UGV  has  to  take  some  action,  such  as 
registering  its  position  relative  to  the  world  m 
relation  to  landmarks,  or  acquiring  new  sensor  data 
for  identifying  likely  obstacles  or  characteristic 
features.  These  positions  are  intermediate  goals  that 
give  the  planner  additional  constraints.  Secondly 
there  are  environmental  constraints  which  will 
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present  obstacles  to  the  UGV  or  prevent  it  from 
moving.  Such  situations  will  include  steep  gradients, 
upstanding  obstacles,  and  holes  in  the  ground.  The 
identification  of  such  environmental  features  is  an 
essential  and  important  task  for  the  sensing  system. 
Thirdly,  there  are  kinematic  constraints.  LGV's  are 
not  omnidirectional  and  their  manoeuvrability  will  be 
determined  by  their  speed  and  steering 
characteristics.  Again,  the  sensing  system  must 
provide  sufficient  warning  in  time  and  distance  to 
allow  for  these  limitations.  In  addition  to  these  three 
basic  constraints,  an  allowance  must  be  made  for 
uncertainty  in  the  UGV's  position.  Sources  of 
uncertainty  range  from  random  errors  in  the  control 
system  to  gross  errors  such  as  wheel  slippage.  The 
local  path  planner  must  take  into  account  control 
based  uncertainty  to  avoid  collisions  by  allowing  for 
an  adequate  "miss"  distance  in  the  planned  trajectory'. 

The  requirement  for  local  path  planning  emphasises 
the  need  for  3D  sensing  of  the  local  environment, 
ideally  out  to  the  limit  of  the  visible  horizon.  In 
ractice,  as  we  shall  see  later,  this  will  be  determined 
y  the  range  capability  of  the  on  board  sensing 
svstem.  A  3D  representation  of  the  scene  ahead  of 
the  vehicle  will  allow  the  local  path  planner  to  choose 
the  "best"  route  (minimum  gradients  and  avoiding 
obstacles)  in  the  direction  of  an  intermediate  or 
ultimate  goal.  An  updated  3D  representation 
obtained  at  a  later  time  when  the  vehicle  has 
advanced  along  the  planned  route  will  enable  a 
revision  to  be  made  to  the  route,  and  so  optimise  the 
next  stage  of  the  route  to  the  goal. 

12.8  3D  Mapping 

We  have  seen  that  3D  mapping  is  necessary  both  for 
route  planning,  and  to  allow  for  the  kinematics  of 
the  vehicle  in  order  to  give  sufficient  advanced 
warning  of  features  which  require  the  vehicle  to 
react.  The  most  direct  means  of  generating  a  3D 
map  is  to  design  a  high  repetition  rate  rangefinder 
system  which  can  scan  an  adequate  field  of  view  in  a 
frame  time  which  is  compatible  with  the  response 
time  of  the  vehicle.  This  implies  a  repetition  rate  of 
many  tens  of  kilohertz,  a  wide  angle  scanning  system 
(typically  30°  x  20°)  and  a  processing  system  which 
can  construct  the  map  from  the  3D  information. 
Taken  together,  this  constitutes  a  formidable 
requirement,  but  systems  have  been  built  at 
considerable  cost  and  complexity  which  have 
provided  the  primary  sensing  facility  in  this  way.  An 
alternative  approach  is  to  provide  a  simpler 
rangefinding  system  which  can  be  used  to 
complement  the  2D  imaging  system  which  is  so 
conveniently'  obtained  from  a  TV  camera.  The 
rangefinding  system  in  this  case  would  assist  in 
resolving  ambiguities  in  the  2D  scene,  such  as 
confirming  or  otherwise  the  existence  of  physical 
discontinuities  indicated  by  spurious  edge  boundaries 
on  die  2D  image.  The  coupling  of  data  from  two 
disparate  sensor  sources  introduces  a  new  topic  for 
study,  namely  data  fusion,  and  this  will  be  discussed 
later. 

We  shall  now  describe  some  of  the  characteristics  of 
the  various  rangefinding  systems  which  have  been 
used  in  mobile  robotics.  These  fall  into  two 
categories:  active  and  passive.  Active  systems 
involve  the  transmission  and  reception  of  pulses  of 
energy  reflected  from  the  object  field  such  as 


ultrasonic,  laser,  and  millimetre  wave  radar.  Passive 
systems  involve  the  processing  of  data  from 
stereoscopic  imaging  sensors  or  optical  flow  sensing. 

Ultrasonic  Sensors  Such  sensors  are  attractive  for 
robotic  applications  because  of  their  low  cost.  A 
typical  device  is  that  manufactured  by  Polaroid  as  a 
camera  rangefinder.  The  range  is  measured  in  terms 
of  the  time  of  flight  of  a  burst  of  ultrasound.  The 
resolution  of  such  a  system  is  limited  by  the 
wavelength  of  the  radiation  used;  at  50kHz  this  is 
about  6mm.  This  in  itself  implies  that  surfaces 
whose  roughness  is  less  than  this  dimension  will  tend 
to  appear  specular  and  hence  targets  may  not  be 
detected  if  the  specular  return  does  not  fall  within  the 
reception  angle  of  the  receiver.  The  simplest 
detection  system  uses  basic  thresholding,  with  time 
varying  gain  to  allow  for  the  decrease  in  received 
signal  at  greater  target  distances  due  both  to 
dispersion  of  the  echo  after  reflection  and  attenuation 
of  ultrasound  in  air.  This  leads  to  maximum 
detection  ranges  of  10m  for  discrete  objects,  and 
considerablv  less  for  objects  in  cluttered 
backgrounds.  The  effective  beam  width  of  the 
Polaroid  transducer  is  about  30°  which  implies  that 
the  accurate  bearing  of  objects  cannot  be  obtained  by 
this  system.  The  receiver  can  be  baffled  to  reduce  its 
angular  reception  angle  by  up  to  50%,  but  this  also 
reduces  the  range  sensitivity  and  increases  the 
sensitivity  to  orientation  effects  of  the  target.  To 
measure  the  angular  position  of  a  target,  it  is 
necessary  either  to  rotate  the  beam  mechanically  or 
to  use  more  than  one  transducer.  Phased  arrays  are 
used  in  radar  and  these  enable  beam  steering  to  be 
done  by  varying  the  phase  at  each  element  in  the 
array;  this  can  be  done  either  in  transmission  or  in 
receive  mode.  The  angular  resolution  of  such  a 
system  is  maximised  by  making  the  separation  of  the 
array  elements  as  large  as  possible;  however,  side 
lobes  are  obtained  unless  the  minimum  separation  is 
of  the  order  of  half  a  wavelength  (3mm  at  50kHz). 
An  eight  element  array  receiver  with  half  wavelength 
separation  has  been  built  (Ref.  20)  giving  a  beanng 
range  of  60°  and  a  resolution  of  6°.  Subsequently  it 
has  been  found  possible  to  use  the  envelopes  of  the 
bursts  of  ultrasound  instead  of  using  the  phase  to 
measure  the  time  delay.  This  eliminates  the  problem 
of  side  lobes  and  has  enabled  four  element  receivers 
to  be  used  with  relatively  large  inter-element  spacing 
(15  wavelengths)  giving"  good  angular  resolution  ana 
requiring  considerably  less  hardware  and  processing 
than  the  half  wavelength  system.  Present  designs 
allow  beam  steering  in  one  direction  only  and 
maximum  sensitivity  to  targets  aligned  at  right  angles 
to  this  direction.  Research  is  currently  aimed  at 
removing  these  restrictions. 

In  summary,  ultrasonic  rangefinders  seem  best  suited 
as  short  range  obstacle  warning  devices,  a  fixed 
array  of  contiguous  rangefinders  giving  "bump  bar" 
protection  at  ranges  of  1  -3  m.  They  are  capable  of 
accurate  range  measurement,  as  we  have  seen,  but 
their  wide  beam  width  makes  them  sensitive  to 
spurious  reflections,  whilst  re-entrant  surfaces  can 
suppress  reflections  and  inclined  targets  can  deflect 
reflections  outside  the  receiver  beam  angle,  rendering 
their  performance  somewhat  unreliable.  They  will  be 
less  affected  by  smoke,  dust  and  ambient  light  levels 
than  optical  systems,  but  their  performance  may  well 
be  degraded  by  rain  or  mud  or  interference  due  to 
noise  and  vibration. 
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Laser  Rangefinders  The  characteristic  feature  of 
the  laser  is  its  narrow  beamwidth  and  this  offers  the 
possibility  of  good  angular  discrimination  in  the 
object  field.  However,  this  brings  with  it  the  need  for 
accurate  synchronisation  with  the  scanning  system 
which  has  to  be  capable  of  covering  a  wide  angular 
field  in  two  directions  in  one  second  or  less.  Range 
is  detennined  by  measuring  the  time  of  flight  between 
the  transmitted  beam  and  the  received  radiation  after 
reflection  from  the  target  surface.  Three  different 
techniques  can  be  employed  to  measure  the  time  of 
flight  (1)  Pulse  detection  which  measures  the  time  of 
flight  of  discrete  pulses,  (2)  Coherent  detection  which 
measures  the  time  of  flight  indirectly  by  measuring 
the  beat  frequency  of  a  frequency-modulated 
continuous  wave  (ftn-cw)  emitted  beam  and  its 
reflection,  and  (3)  Direct  detection,  which  measures 
the  time  of  flight  indirectly  by  measuring  the  shift  in 
phase  between  an  amplitude-modulated  continuous 
wave  (am-cw)  emitted  beam  and  its  reflection. 
Experimental  systems  using  all  three  techniques  have 
been  built,  but  most  systems  developed  as  imaging 
laser  radars  have  been  based  on  pulse  detection  or 
am-cw.  For  both  such  systems  the  GaAs  semi¬ 
conductor  laser  operating  in  the  near  infra-red  wave 
band  (800-900  nanometres)  is  capable  of  modulation 
at  repetition  rates  of  typically  25kHz.  Peak  pulse 
powers  of  lOwatts  for  pulse  detection  and  mean 
output  powers  in  the  range  100-300mw  for  am-cw 
have  been  reported  (Refs. 2 1,22).  In  the  pulse 
detection  system  (Ref.21)  the  rangefinder  was 
scanned  in  a  raster  over  a  40°x30°  field  of  view  by 
two  rotating  coaxial  polygons.  The  laser  was  fired  in 
synchronism  with  the  scan  and  measured  the  range  to 
points  in  the  scene  at  equal  angular  intervals  of 
approximately  0.5°,  giving  a  56x55  pixel  display. 
The  range  resolution  was  2.5cm  (s.d.)  measured 
against  a  normal  target  in  the  measurement  range  6m 
to  50m.  The  sequence  of  operations  applied  to  the 
raw  range  data  was  as  follows 

(1)  Scan  conversion;  a  look  up  table  reordered 
the  pixel  data  obtained  from  the  scanner. 

(2)  The  slope  (relative  to  the  vertical)  of  each 
triangular  elemental  area  defined  by  3  neighbouring 
pixels  was  determined. 

(3)  Region  growing  was  carried  out  to  associate 
clusters  of  elemental  areas  with  slopes  greater  than  a 
critical  value  (defining  an  obstacle  for  the  vehicle  in 
question).  Neighbouring  areas  were  clustered  if  their 
range  separation  was  less  than  a  preset  value. 

(4)  A  simple  obstacle  description  was  obtained 
by  enclosing  the  obstacle  clusters  m  boxes. 

(5)  The  pathlength  to  the  nearest  box-obstacle 
along  a  given  bearing  from  the  sensor  was  supplied 
to  the  navigation  subsystem. 

The  system  demonstrated  the  principle  of  detecting 
upstanding  obstacles  in  the  path  of  the  vehicle,  but 
two  problems  became  evident  as  a  result  of  this 
work.  In  order  to  achieve  a  useful  maximum  range 
from  the  system,  the  collecting  area  of  the  receiver 
needed  to  be  several  square  centimetres.  With  the 
scanner  mechanism  operating  in  the  object  field,  this 
implied  that  the  scanning  system  of  rotating  polygons 
was  both  large  and  heavy  (3  0x3 0x3 0cm3). 
Alternative  oscillating  plane  mirror  systems  can  be 


equally  cumbersome  (Ref.22).  Secondly,  the  pixel 
points  in  the  image  were  scanned  by  the  polygon 
system  in  a  fixed  hut  random  sequence.  When  the 
vehicle  platform  was  in  motion  this  led  to  "image 
smear"  due  to  the  processing  of  adjacent  pixels 
whose  data  was  gathered  at  different  time  intervals. 

We  can  conclude,  therefore,  that  laser  techniques 
can  produce  accurate  and  precise  directional  range 
image  data  out  to  ranges  of  50m  depending  on  target 
reflectivity.  Mechanical  scanning  techniques  enable 
wide  angle  fields  of  view  to  be  scanned,  but  the 
design  of  wide  angle  two  dimensional  scanning 
systems  suitable  for  operation  on  vehicles  in  field 
environments  is  not  a  trivial  problem. 

Millimetre  Wave  Ranging.  Millimetre  wave 
techniques  do  not  suffer  many  of  the  environmental 
difficulties  which  beset  ultrasonic  and  laser  ranging 
such  as  smoke,  fog  and  mud.  However,  with 
frequencies  in  the  GHz  band,  problems  centre  on 
beamwidth,  accuracy  of  ranging  and  ability  to  range 
effectively  on  non-metallic  objects  as  well  as  metallic 
objects  due  to  relative  absorption.  The  technique  is 
ideally  suited  to  detecting  and  tracking  vehicles 
ahead  of  the  vehicle  having  the  installed  radar, 
because  the  large  radar  cross  section  of  metallic 
vehicles  (10- 15m2  at  94GHz)  can  ensure  strong 
returns  from  distances  well  beyond  300m.  However, 
for  the  route  finding  and  navigation  task  ranging  to 
natural  objects  is  required  ana,  as  these  are  mostly 
non-metallic,  detection  is  much  more  problematic, 
with  radar  cross-sections  for  miscellaneous  road 
debris  being  typically  0.  lm2.  By  comparison,  the 
cross-section  or  a  road  surface  is  <104nr  per  square 
metre  of  road  area  which  is  sufficiently  low  to  offer 
the  possibility  of  detecting  debris  lying  on  a  prepared 
road  surface  (Ref.23) 

Modem  solid  state  technology  has  solved  the 
problem  of  the  generation  of  power  at  millimetre 
frequencies,  the  maximum  power  for  human  safety 
of  lOmw  being  easily  generated  by  a  solid-state 
Gunn  diode  oscillator  at  80GHz.  The  easiest 
modulation  required  to  obtain  range  resolution  from 
a  cw  source  is  frequency  modulated  continuous  wave 
(ftn-cw).  The  transmitter  frequency  is  varied  linearly 
with  time  at  a  "chirp"  rate  a  (Fig.  20),  and  the 
transmitted  and  received  frequencies  are  mixed 
together  in  the  receiver  mixer  to  generate  a  constant 
beat  frequency.  If  the  time  delay  from  transmission 
to  reception  is  x  and  the  rate  of  change  of  transmitter 
frequency  with  time  is  a,  then  the  beat  frequency  fb 
will  be  fb=ax  (Fig.20).  If  the  range  is  r,  then  x-2r/c 
where  c  is  the  speed  of  propagation.  The  beat 
frequency  is  thus  fb  =  2ar/c  so  the  range  to  the  target 
can  be  found  by  measuring  the  beat  frequency.  It 
can  be  shown  that  the  range  resolution  is  dependent 
on  the  bandwidth  over  which  transmission  occurs 
and,  for  car  obstacle  avoidance  radars,  this  will 
typically  be  about  150MHz  to  give  a  range 
resolution  of  lm  with  a  period  of  about  1ms.  The 
beat  frequency  from  a  target  at  a  range  of  250m  will 
be  250kHz.  If  this  is  the  maximum  range  of  interest 
then  the  receiver's  IF  bandwidth  will  be  between 
1kHz  and  250kHz.  The  processing  is  ideally  suited 
to  the  use  of  a  digital  signal  processor  (DSP)  chip  to 
perform  a  fast  Fourier  transform  (FFT)  on  the  IF 
signal. 
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Fig. 20  Principle  of  FMCW  radar 

There  remains  the  question  of  receiver  dimensions. 
This  will  be  dependent  on  the  transmitter  wavelength 
and  the  beam  divergence  required.  At  a  frequency  of 
80GHz  (wavelength  4mm)  the  approximate  antenna 
width  for  a  1  degree  beamwidth  is  25cm  which  is  still 
quite  large  for  a  convenient  installation.  Scanning 
presents  a  further  mechanical  problem  but,  of  course, 
the  need  for  this  can  be  sacrificed  at  the  expense  of  a 
wider  beamwidth. 

In  summary  then,  modem  technology  is  now  bringing 
nearer  the  feasibility  of  millimetre  wave  radar  for 
cost  effective  obstacle  detection,  and  intelligent 
cruise  control  with  which  a  vehicle  can  maintain  a 
safe  distance  behind  the  vehicle  in  front  on  the  open 
highway.  However,  it  is  not  clear  at  present  whetner 
such  systems  can  assist  the  navigation  of  UGV's  in 
the  cross  country  environment. 

3D  Mapping  by  Passive  Vision  Shape  can  be 
perceived  by  comparing  the  images  of  a  scene 
observed  from  different  perspectives.  This  is  most 
obviously  implemented  by  using  binocular  stereo, 
depth  being  deduced  by  triangulating  over  the  stereo 
baseline.  Practical  constraints  tend  to  limit  the 
length  of  the  base  line  and  hence  the  accuracy  of 
depth  measurement.  Maintaining  correct 
registration,  and  stable  calibration,  are  other 
problems  which  have  to  be  faced.  On  the  other  hand, 
it  is  also  possible  to  make  use  of  object  or  sensor 
motion  to  obtain  multiple  views  with  a  potentially 
longer  base  line  for  triangulation.  This  can  be  done 
by  capturing  sequences  of  images  from  a  single 
moving  camera  and  generating  structure  from  motion 
by  appropriate  processing.  This  is  the  basis  of  the 
DROID  system  developed  at  Roke  Manor  Research 
Ltd.  (Ref.  24).  The  concept  depends  on  identifying 
features  or  tokens  in  a  video  frame  and  tracking  the 
corresponding  features  in  subsequent  frames.  It  is 
important  to  avoid  ambiguity  in  matching  the 
features  between  frames,  by  locating  the  features 
reliably  and  as  precisely  as  possible.  This  is  carried 
out  by  a  comer  detector  algorithm  which  locates 
vertices  of  objects  and  spots  which  occur  in  the 
images  of  the  viewed  scene.  Typically  up  to  400 
comers  are  detected  within  a  single  256x256  pixel 


image.  Determination  of  the  correspondence  of  the 
current  frame  with  the  emerging  3D  representation 
through  feature  matching,  enables  the  camera 
motion  estimate  to  be  refined.  This  is  important 
since  it  is  a  knowledge  of  the  camera  motion  between 
frames  which  determines  the  ultimate  accuracy  of  the 
range  distance  data.  The  3D  representation  of  the 
surface  structure  is  developed  using  Delaunay 
triangulation.  This  allows  a  compact  set  of 
triangular  facets  to  be  generated  in  the  image  plane 
using  the  set  of  the  most  reliable  3D  points,  each 
triangular  facet  being  single  valued  in  depth  from  the 
camera.  This  representation  is  then  used  to  construct 
a  set  of  surface  contours,  typically  lm  apart  parallel 
to  the  road  and  with  transverse  contours  at  a  spacing 
of  3m.  Finally,  the  contour  map  is  segmented  into 
regions  which  are  traversable  for  the  vehicle  and 
those  which  are  not  traversable.  In  practice,  the 
addition  of  a  second  "stereo"  camera  improves  the 
accuracy  of  measurement  of  the  camera  ego  motion 
as  well  "as  providing  some  3D  information  when  the 
vehicle  platform  is  stationary.  This  technique 
involves  a  heavy  computational  load,  but  real  time 
operation  is  now  becoming  feasible  and  should  allow 
the  potential  to  be  realised  in  practice. 

Whilst  the  DROID  technique  offers  a  powerful 
passive  means  of  generating  a  3D  representation  of  a 
static  scene,  there  remains  the  problem  of  detecting 
and  tracking  moving  objects  in  the  field  of  view.  A 
recent  development  uses  optical  flow  measurement  to 
distinguish  object  movement  relative  to  the 
surroundings  as  seen  from  a  moving  platform 
(Ref.25).  The  system  is  termed  ASSET  (A  Scene 
Segmenter  Establishing  Tracking)  and  again  depends 
on  feature  based  matching  between  frames  in  the 
image  plane.  This  is  then  followed  by  object 
segmentation  and  tracking.  Each  frame  taken  by  the 
video  camera  is  initially  processed  to  find  two 
dimensional  features  and  edges.  This  is  a  key  factor 
in  the  process  and,  in  order  to  match  features 
between  frames,  both  image  brightness  and  local 
image  spatial  derivatives  are  used.  These  image 
functions  are  combined  into  a  vector  which  is 
compared  with  the  corresponding  vector  in  the 
second  image.  The  flow  vectors  are  next  segmented 
into  regions  which  have  consistent  internal  now  and 
which  are  different  from  each  other.  This  makes  the 
reasonable  assumption  that  the  separately  moving 
image  regions  have  "roughly  constant"  image  flow 
which  is  different  from  the  flow  of  adjacent  regions. 
Having  clustered  the  image  flow  vectors,  the  outline 
of  each  cluster  is  found  using  the  outlving  feature 
points  of  the  cluster.  If  overlap  is  found,  the  cluster 
which  will  be  occluded  is  predicted  using  the  safe 
assumption  for  ground  based  situations  that  this 
cluster  will  have  a  higher  image  position  for  its 
lowest  point  than  will  the  other  cluster.  Fine  tuning 
of  the  cluster  boundaries  is  then  carried  out  using 
edges  from  the  original  image.  The  ASSET 
technique  has  proved  successful  in  dealing  with 
scenes  involving  the  viewing  of  overtaking  vehicles 
from  a  following  vehicle.  It  requires  no  camera 
calibration  or  knowledge  of  the  camera  motion.  The 
next  stage  is  the  integration  of  ASSET  with  the 
structurc-from-motion  system  which  recovers  world 
structure  in  a  static  environment.  ASSET  will  be 
used  to  segment  out  and  track  moving  vehicles,  and 
the  static  part  of  the  scene  will  then  be  tracked  by  the 
structure-ffom-motion  system. 
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12.9  Data  Fusion 

In  order  for  a  mission  to  be  completed  successfully 
the  correctness  of  decision  making  en  route  must  be 
of  a  very  high  order.  This  is  particularly  necessary 
in  the  case  of  route  finding  and  obstacle  detection 
and,  as  we  have  seen,  no  one  technique  will  provide 
the  ultimate  solution  to  the  problem.  Several 
possible  sources  of  information  are  available  in  the 
form  of  sensor  data,  digital  map  data  or  other  forms 
of  prior  knowledge.  The  UGV  control  system  has 
been  considered  to  be  made  up  of  a  number  of 
processing  levels  (Fig.9).  The  lowest  levels  process 
the  raw  sensor  data  to  extract  information,  whilst  the 
higher  levels  attempt  to  reason  with  the  extracted 
information  in  a  more  symbolic  manner.  Data  fusion 
can  be  considered  within  or  bridging  these  levels  by 
aggregation  of  information  and  representation  at  a 
higher  level.  A  specific  example  is  the  case  of  low 
level  data  processing  for  UGV  guidance  along  a 
road.  The  raw  data  will  comprise  colour  television 
images  and  the  requirement  is  reliably  to  determine 
the  road  location.  The  images  can  be  processed  to 
extract  candidate  road  surface  using  colour,  texture 
and  3D  processing.  None  of  these  techniques  will  be 
totally  reliable  on  their  own,  but  data  fusion  applied 
in  an  effective  way  should  improve  the  situation. 

There  are  several  possible  approaches  including 
statistical  inference,  fuzzy  logic,  rule-based  systems, 
Kalman  estimation,  neural  networks  and  others 
(Ref.  26).  Statistical  inference  methods  allow  the 
combination  of  information  if  the  appropriate 
probabilities  can  be  obtained,  for  example  by 
training.  A  tree  of  hypotheses  is  set  up  concerning 
the  label  (e.g.  road,  verge,  sky,  other)  which  should 
be  attached  to  each  picture  element  in  the  sensor 
image.  Each  image  processing  module  then  delivers 
evidence,  for  example,  in  the  form  of  a  probability, 
which  is  appropriate  to  some  level  in  the  tree,  and 
which  concerns  the  label  to  be  attached  to  the  picture 
element.  The  probabilities  can  be  combined 
according  to  Bayes'  rule  to  remain  consistent  across 
the  label  tree,  and  the  combined  probabilities  are 
used  to  decide  on  the  most  appropriate  labelling  of 
the  image. 

To  illustrate  how  Bayes'  decision  theory  can  be 
applied  to  fuse  information,  consider  a  simple  case 
where  there  is  assumed  to  be  complete  statistical 
knowledge  of  the  process.  It  is  required  to  determine 
P(sj|X)  which  is  the  conditional  probability'  that  the 
labelling  (state  of  a  nature)  at  a  pixel  site  is  Sj  given  a 
set  of  measured  features  X={xi,x2,...,xn} 

Direct  determination  of  the  probability  P(sj  I X)  does 
not  accord  well  with  human  experience  and  may  be 
ill-defined  because  of  noise,  or  otherwise  ill- 
conditioned.  However,  Bayes'  rule  may  be  used  to 
split  P(sjlX)  into  two  parts  which  depend  on  the 
available  measurements  and  prior  knowledge  of  the 
labelling.  Thus, 

P(sJ|X)  =  P£X^)P(si) 

where  P(X|s>)  =  probability  of  a  measured  feature 
X  given  that  the  label  is  Sj. 

P(Si)  =  prior  probability  that  the  label  is  Sj 

p(X)=£p(x|sj)P(sj) 

j 


If  the  P(X  j  sj)  are  known  or  can  be  estimated  from 
training  data,  then  they  may  be  combined  with  prior 
knowledge,  or  an  estimate  of  P(s,)  to  provide  a 
maximum  a-  posteriori  estimate  of  Sj.  This  form  of 
approach  has  been  applied  to  processing  colour 
images  for  autonomous  vehicle  guidance  (Ref. 2 7). 
Here  the  states  of  nature  comprised  "road"  and  "non¬ 
road"  and  the  feature  vector  consisted  of  6  colour 
components  from  two  colour  cameras.  A  number  of 
Gaussian  models  were  assumed  for  the  colour  of 
road  and  non-road  areas,  and  the  parameters  of  their 
class  conditional  densities  P(X  1  sj)  were  initially 
determined  by  supervised  learning  and  then 
iteratively  updated.  P(sj)  was  estimated  as  the  ratio 
of  the  number  of  pixels  in  Sj  to  the  total  number  of 
image  pixels. 

13.  COMPUTING  TECHNOLOGY 

Much  of  the  low  level  processing  that  has  been 
described  is  very  computationally  intensive,  and  the 

3uest  for  real  time  operation  focuses  the  attention  on 
ie  development  of  computing  power  which  will 
realise  this  in  practice.  In  order  to  obtain  a 
benchmark  for  the  computing  pow  er  needed,  we  can 
consider  a  basic  convolution  operation  on  a  frame  of 
video  data.  For  an  n  x  n  convolution  over  an  image 
of  N  x  M  pixels  the  number  of  operations  P  per  sec 
at  a  frame  rate  of  f/sec  is  given  by  P=n2NMf. 

Substituting  typical  values  of  n=5,  N=M--5 1 2  and 
f=25  we  obtain  P=164xl06  operations  per  second  or 
164  Mops.  We  have  seen  that  3D  mapping,  using 
DROID  for  instance,  involves  a  sequence  of 
processes,  namely  (1)  comer  detection,  (2)  comer 
matching,  (3)  3D  calculation,  (4)  contour  fitting. 
This  is  followed  by  higher  level  reasoning  about  the 
traversability  of  the  contour  map.  Thus  at  least  5 
levels  of  processing  are  required  before  control 
instructions  can  be  issued  to  the  vehicle  controller. 
Of  course,  the  higher  levels  of  processing  will  be 
operating  at  lower  effective  frame  rates  (vehicle 
response  time  typically  lOPIz  or  less)  and  smaller 
units  of  data  than  the  pixel  level  will  then  be  involved 
but,  nevertheless,  operations  at  1000  Mops  per  video 
frame  can  be  envisaged  as  a  commonplace 
requirement.  Fortunately,  much  of  the  lower  level 
processing  is  well  suited  to  parallel  operations  which 
matches  well  to  modem  computer  architectures. 

The  development  of  microprocessor  chips  has  led  to 
computing  power  being  doubled  every  two  years  over 
the  period  1985  to  1993.  Currently  the  Intel  i860 
vector  processor  operates  at  80MFLOPS  (Floating 
Point  Operations  per  second)  and  the  T9000 
transputer  at  25  MFLOPS.  Of  the  Sun  family  of 
workstations,  the  SPARC  10  machine  introduced  in 
1993  operates  at  10  MFLOPS.  The  Intel  80586 
microprocessor  which  appeared  in  1 993  uses  a  clock 
speed  of  50MHz,  and  plans  have  been  announced  for 
a  2000MIPS  (instructions  per  sec)  microprocessor  to 
be  introduced  in  the  year  2000  (tne  80886?) 
operating  with  a  clock  speed  of  250  MHz.  This 

Setition  has  led  Motorola  to  leak  plans  for  a 
MIPS  reduced-instruction-set  microprocessor 
as  their  state  of  the  art,  year-2000  microprocessor. 

Parallel  computers,  typified  by  the  WARP  I  machine 
which  is  in  use  on  the  Carnegie  Mellon  University 
Navlab  vehicle  testbed,  first  appeared  in  1985  with  a 
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capability  of  80  MFLOPS.  The  WARP  II  then 
achieved  1200  MFLOPS  in  1990. 

By  comparison.  Symbolic  Processors  have  seen  a 
similar  escalation  m  performance  with  time.  Their 
computing  speed  is  measured  by  the  rate  of  logical 
inferences  which  they  carry  out  per  second  (LIPS). 
A  logical  inference  operation  makes  the  deduction 
that  "If  A,  then  B".  In  1988  a  portable  ruggedised 
LISP  machine  was  operating  at  10  MLIPS,  whilst 
the  Connection  Machine  with  65,000  processors  was 
achieving  nearly  1000  MLIPS. 

In  the  area  of  dynamic  random  access  memories 
(DRAMs),  progress  is  expected  to  continue  at  the 
same  rate  as  that  realised  over  the  past  25  years  with 
DRAM  chip  densities  quadrupling  every  3  years. 
64Mbit  DRAMs  were  scheduled  for  sample 
introduction  in  1993,  256  Mbit  DRAMs  are  planned 
for  1996,  and  1000  Mbit  DRAMs  should  make  their 
sample-quantity  debut  around  the  year  2000. 

Thus  there  is  no  sign  at  present  that  the  increasing 
availability  of  computiag  power  is  about  to  tail  off 
and,  therefore,  significant  advances  in  intelligent 
decision  making  for  autonomous  vehicles  can  be 
anticipated  in  the  near  future. 

14.  AUTONOMOUS  GROUND  VEHICLE 
TEST  BEDS 

Research  on  autonomous  ground  vehicles  has  led  to 
the  development  of  a  number  of  vehicle  test  beds. 
Some  are  laboratory  based,  others  project  based. 
The  VaMoRs  vehicle  at  the  Armed  Forces  University 
at  Munich,  ROVA  at  DRA  Chertsey,  DARDS  at 
SAGEM  in  France,  and  the  Navlab  at  Carnegie 
Mellon  University  are  examples  of  laboratory  based 
vehicles.  The  PANORAMA  project,  a  part  of  the 
European  Esprit  Programme  has  also  generated 
autonomous  research  vehicles,  and  another  European 
initiative  PROMETHEUS  has  done  likewise.  The 
PROMETHEUS  project  was  started  in  1986  and  is 
jointly  funded  by  Government  and  the  European 
Motor  Manufacturers.  One  of  its  objectives  is  to 
improve  safety  on  the  public  highways  of  the  future. 
To  this  end  intelligent  cruise  control,  collision 
avoidance  and  dual  mode  route  guidance  are  research 
topics  which  impinge  on  autonomous  navigation,  and 
many  of  the  collaborating  nations  have  test  vehicles 
to  demonstrate  their  achievements  in  these  research 
areas. 

Two  systems  are  now  described  in  more  detail  as 
they  provide  examples  of  total  vehicles  embodying 
many  of  the  subsystems  and  technologies  which  have 
been  discussed  earlier.  The  two  projects  are  the 
PANORAMA  ESPRIT  II  (Ref.28)  and  the  Navlab 
(Ref.  29). 

Panorama.  The  project  aims  to  develop  a  perception 
and  navigation  system  for  autonomous  outdoor 
mobile  robots  and:  is  built  around  a  4  x  4  road 
vehicle.  A  hierarchical  approach  was  chosen  similar 
to  the  NASREM  architecture  (Fig.  11)  with  the 
obstacle  avoidance  loop  comprising  a  fully  integrated 
system  coupling  higher  levels  (planner)  with  lower 
levels  (data  acquisition,  actuation).  The  system  has 
been  tuned  to  work  at  low  speed  (<5m/sec)  on 
relatively  even  terrain,  but  its  generic  design  will 
allow  extension  to  more  demanding  constraints.  The 


sensor  system  is  based  on  scanning  rangefinders, 
ultrasonic  for  ranges  up  to  5m,  and  laser  for  ranges 
up  to  50m.  The  laser  beam  scans  at  1kHz  in  azimuth 
only,  whilst  on  the  move,  and  delivers  measurements 
indicating  tic?  or  occupied  sectors.  The 
environmental  modelling  produces  a  probabilistic 
occupancy  grid  having  typically  128  x  128  cells  each 
50cm  x  50cm.  Updating  from  a  single  scan  takes 
around  10  -  20  ms.  The  local  path  planner  has  the 
task  of  directing  the  vehicle  on  the  planned  route 
whilst  avoiding  unexpected  but  perceived  obstacles. 
It  provides  continuously  an  updated  trajectory  for  the 

Ei  toting  system  computed  with  geometric  and 
inematic  constraints.  At  the  end  of  the  obstacle 
avoidance  loop  is  the  Piloting  System  whose  purpose 
is  to  follow  the  instructions  of  tne  local  path  planner. 
This  employs  a  hierarchical  architecture  with  four 
separate  modules  (1)  a  trajectory  generator  [a  servo 
loop  on  trajectory  and  goal  position  at  0.1  -  0.5Hz] 
(2)  a  trajectory  follower  [a  servo  loop  on 
position/orientation  at  4Hz]  (3)  a  kinematic 
controller  fa  servo  loop  on  speed/heading  at  20  Hz] 
and  (4)  a  dynamic  controller  [a  servo  loop  on  throttle 
valve  angle/brake  pressure/steering  angle  at  lOOHzj. 


generate  motion  reactions,  as  all  obstacles  are  not 
necessarily  in  the  planned  corridor.  Therefore,  fine 
piloting  control  is  possible  with  smooth  obstacle 
avoidance  while  moving.  Also  the  environmental 
modelling  by  grid  representation  is  generic  enough  to 
allow  efficient  fusion  of  information  coming  from  a 
priori  knowledge  and  sensors  (lasers,  ultrasonics, 
vision). 

Carnegie  Mellon  Navlab.  The  Navlab  is  a  testbed 
for  research  in  outdoor  navigation,  image 
understanding  and  the  role  of  human  interaction  with 
intelligent  systems.  The  vehicle  was  first  conceived 
in  1985  and  is  based  on  a  General  Motors  van  whose 
final  all-up  weight  has  reached  5.3tons.  The  length 
of  the  venicle  is  5.5m  and  the  minimum  turning 
radius  7.5m.  The  Navlab  controller  manages  all 
locomotion.  It  interacts  with  a  computer  host  and 
human  operator  to  implement  various  degrees  of 
autonomy.  The  controller  queues  and  executes 
commands  originating  from  a  computer  or  human 
host,  providing  a  "Virtual  vehicle  interface  that 
hides  details  of  the  actual  hardware  and  therefore 
facilitates  adaptation  to  future  navigation  testbeds. 
The  control  computer  accepts  commands  from  a  host 
or  human  operator  who  can  intervene  at  various 
levels  of  control  to  ensure  safe  operation  during 
experiments.  It  was  assumed  that,  during 
development,  the  higher  levels  of  computing  would 
not  succeed  in  all  situations.  The  control  computing 
therefore  provides  a  graceful  transition  between 
computer  and  human  control  when  failures  occur. 
The  human  input  facility  is  also  useful  for  set  up  and 
recovery  during  experiments.  The  sensing  system 
provides  for  vision,  laser  and  ultrasonic  ranging.  A 
fixed  video  camera  system  is  mounted  in  the  front  of 
the  vehicle  above  the  driving  cab  and  provides  a  wide 
angle  view  of  the  scene.  The  current  laser  ranging 
device,  manufactured  by  ERIM,  operates  at  a 
wavelength  of  900  nanometres  and  produces  a 
256  x  64  x  8  bit  depth  map  at  2  frames  per  second. 
Each  sensor  commonly  requires  its  own  workstation 
or  specialised  processor.  Another  computer  runs  the 
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blackboard  system  that  integrates  perception, 
modelling  and  planning. 

The  Navlab  project  seeks  to  build  complete 
autonomous  systems  capable  of  outdoor  navigation, 
both  on  roads  and  cross-country.  The  biggest 
challenge  has  been  in  the  area  of  image 
understanding  in  difficult  conditions  with  the 
emphasis  on  unstructured  roads,  on  the  changing 
appearance  of  structured  roads  in  dappled  shadows 
and  at  intersections,  and  on  off-road  navigation  over 
rough  terrain.  The  Navlab  vehicle  has  driven 
autonomously  at  slow  speeds  along  unmarked, 
unmapped  trails,  locating  and  traversing 
intersections.  On  more  typical  structured  roads  the 
vehicle  has  driven  up  to  its  mechanical  limit  of 
28kph.  It  can  run  without  a  map  or  use  maps  it  has 
built,  along  with  information  from  previous  runs,  to 
select  different  behaviours  at  different  locations.  Off 
road  the  Navlab  can  move  slowly  over  moderately 
rough  terrain  and  can  map  large  areas  as  it  drives. 

The  current  research  programme  is  addressing  two 
particular  areas:  machme  learning  and  human- 
computer  interaction  (Ref.  15).  Machine  learning  is 
not  directed  at  building  new  capabilities  but  at 
automatically  improving  the  performance  of  existing 
systems.  The  work  on  human-computer  interaction 
(HCI)  is  to  fill  gaps  in  the  operating  profile  where  the 
vehicle  cannot  reliably  run  autonomously.  STRIPE 
has  already  been  described  (Section  11)  and  is  an 
example  of  the  HCI  thrust  on  supervised 
teleoperation.  ALVINN  (Section  12.4)  is  an 
example  of  machine  learning  using  neural  nets. 

More  recently  (Ref.  15)  machine  learning  has  been 
used  to  improve  a  feature  based  trajectory  vision 
system.  This  system  is  the  YARF  (Yet  Another 
Road  Follower)  which  has  specialised  trackers  for 
finding  yellow  lines,  white  lines  and  other  scene 
features.  It  finds  the  features  in  one  image,  updates 
its  model  of  road  location  and  shape,  and  predicts 
where  the  features  should  appear  in  the  next  image. 
The  individual  feature  trackers  are  good  but  not 
perfect.  There  are  several  areas  where  machine 
learning  could  be  used  to  improve  YARF's 
performance  of  tracking  white  and  yellow  lines.  The 
road  follower  Chameleon  is  being  built  to  address 
two  separate  problems:  the  confidence  problem  and 
the  location  problem.  The  first  experiment  is  to  learn 
confidence  measures  for  deciding  whether  lines  are 
present  in  the  predicted  tracking  windows.  In  the 
second  experiment,  the  location  estimation  of  the 
features  is  being  tackled.  In  both  cases  a  supervised 
neural  network,  trained  by  back  propagation,  is  used. 
This  should  enable  a  better  understanding  of 
relatively  well  defined  image  features  to  be  obtained, 
rather  than  attempting  to  solve  the  overall  driving 
problem  using  a  single  artificial  neural  network  as  in 
ALVINN. 

The  Navlab  will  be  embodying  these  techniques  as 
they  are  proved.  Already  supervised  teleoperation 
using  STRIPE  can  be  switched  to  autonomous 
driving  during  a  run.  Even  while  driving  with 
STRIPE  the  automatic  obstacle  detection  system  can 
be  kept  running  to  provide  a  "soft  bumper".  This 
increases  safety  and  robustness,  while  decreasing  the 
operator  workload. 


The  Navlab  project  is  an  excellent  example  of 
fundamental  research  being  directed  at  the  practical 
problems  of  autonomy,  and  demonstrating  the  results 
of  this  research  on  a  fullscale  test  bed  which  has  the 
potential  for  further  extension  of  its  capability  for 
some  years  to  come. 
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SUMMARY 

This  paper  presents  human  factors  considerations  of  remote 
manipulation,  sometimes  called  teleoperation.  The  paper  reviews 
two  broad  classes  of  sensing  and  display  considerations,  namely 
(1)  vision  and  video  feedback,  and  (2)  haptic  sensing  and 
display,  including  force  and  touch  feedback  from  remote 
manipulators.  Next  “telepresence”  and  virtual  environments  are 
discussed  in  relation  to  one  another.  The  paper  then  discusses 
two  broad  classes  of  teleoperator  control,  namely  (1)  direct 
manual  control  and  (2)  supervisory  control.  Finally  the  paper 
considers  briefly  the  subjects  of  cognition  and  mental  models  as 
they  relate  to  remote  manipulation. 

This  material  is  drawn  largely  from  reference  [1],  a  recent  book 
on  teleoperation,  automation  and  supervisory  control  by  the 
author.  Many  sentences  and  paragraphs  are  direct  quotes  from 
that  book,  except  for  reference  and  figure  numbers  (hence  quotes 
are  not  used).  The  organization  of  the  material  differs. 

INTRODUCTION 

Some  Definitions  of  “Tele”  Terms 

The  following  are  not  the  only  terms  which  may  be  new  to  the 
reader  not  already  familiar  with  the  field;  others  will  be 
introduced  as  needed.  This  set  of  definitions  of  “tele”  terms,  in 
modified  form  from  reference  [1],  is  presented  here  to  emphasize 
the  special  human  communication  aspects  of  remote 
manipulation. 

A  teleoperator  is  a  machine  that  extends  a  person’s  sensing  and/or 
manipulating  capability  to  a  location  remote  from  that  person.  A 
teleoperator  typically  includes:  artificial  sensors  of  the 
environment;  artificial  arms  and  hands  or  other  devices  to  apply 
forces  and  perform  mechanical  work  on  the  environment;  a 
platform  or  vehicle  for  supporting  or  moving  these  in  the  remote 
environment;  and  communication  channels  to  and  from  the 
human  operator. 

Telemanipulation  is  sometimes  used  as  a  synonym  for 
teleoperation,  unless  one  means  remote  control  of  a  vehicle  for 
inspection  only  .The  term  teleoperation  refers  most  commonly  to 
direct  and  continuous  human  control  of  the  teleoperator,  but  can 
also  be  used  generically  (and  literally  —  i.e., “operating  at  a 
distance”)  to  encompass  telerobotics  . 

A  telerobot  is  an  advanced  form  of  teleoperator  which  a  human 
operator  supervises,  or  on  which  he  employs  supervisory 
control..  That  is,  the  operator  intermittently  communicates  to  a 
computer  information  about  goals,  constraints,  plans, 
contingencies,  assumptions,  suggestions,  and  orders  relative  to  a 
remote  task,  getting  back  integrated  information  about 
accomplishments,  difficulties,  and  concerns  and  (as  requested) 
raw  sensory  data.  The  subordinate  telerobot  executes  the  task  on 
the  basis  of  information  received  from  the  human  operator  plus 
its  own  artificial  sensing  and  intelligence.  The  term  supervisory 
control  is  commonly  used  to  refer  to  human  supervision  of  any 
semi-autonomous  system  (an  aircraft,  a  chemical  or  power  plant. 


etc.)  regardless  of  the  distance  separating  it  from  its  human 
operators),  while  the  term  telerobotics  commonly  refers  to 
supervisory  control  of  a  teleoperator  (a  machine  that  is  remote 
from  the  operator). 

An  anthropomorphic  teleoperator  or  telerobot  has  a  human-like 
form,  in  that  it  senses  its  environment  with  what  resemble  eyes, 
manipulates  mechanical  objects  with  what  resemble  arms  and 
hands,  and/or  moves  in  many  directions  with  what  resemble 
human  body  motions.  Thus  the  anthropomoiphic  teleoperator  or 
telerobot  provides  the  human  operator  with  an  important  remote 
body  image,  which  the  non-anthropomorphic  teleoperator  or 
telerobot  does  not. 

Teleproprioception  refers  to  the  human  operator’s  sensing  and 
keeping  track  of  the  location  and  orientation  of  the  teleoperator 
and  its  arms  and  hands  relative  to  its  base,  to  each  other,  to 
external  objects,  and  to  the  location  of  the  operator’s  body,  arms, 
and  hands.  The  closely  associated  term  telekinesthesis  refers  to 
the  operator’s  ability  to  identify  the  dynamic  movements  of  the 
teleoperator  and  its  arms  and  hands  relative  to  its  base,  to  each 
other,  to  external  objects,  and  to  the  velocity  or  forces  imposed 
by  the  operator’s  body,  arms,  and  hands. 

Telepresence  refers  to  a  person’s  experiencing  a  state  of  being 
present  in  an  environment  other  than  where  the  person  actually  is. 


VISUAL  SENSING  AND  DISPLAY 

Comparison  of  direct  vision  vs.  closed  circuit  video 

The  most  important  sensory  communiction  channel  for  remote 
manipulation  is  vision.  For  some  teleoperation  tasks  which  are 
not  too  distant,  e.g.,  viewing  radioactive  objects  through  leaded 
glass  windows,  peering  out  through  manned  submersible 
portholes,  viewing  near  space  telemanipulations  from  within 
pressurized  space  vehicles,  direct  viewing  is  preferred  over 
video.  Whether  or  not  this  is  wise  depends  upon  how  close  the 
video  camera  can  be  brought  to  the  objects  of  interest,  whether 
the  camera  has  pan,  tilt,  or  translational  capability,  what  are  its 
contrast  and  illumination  ranges  and  frame  rate,  how  well  it 
reproduces  color,  and  how  important  any  of  these  factors  is  for 
the  job  to  be  done. 

Massimino  and  Sheridan  [2]  compared  telemanipulation 
capability  for  direct  vision  vs.  video  in  simple  block-insertion 
tasks.  They  found  that  mean  task-completion  times  dereased 
dramatically  as  the  subtended  angle  of  the  critical  objects  in  the 
visual  field  increased  beyond  1°  and  the  frame  rate  increased 
beyond  3  frames  per  second  (Figure  1).  However,  for  broadcast 
standard  resolution  there  was  no  significant  difference  between 
direct  viewing  and  video  when  the  total  visual  field  of  objects  to 
be  manipulated  was  the  same. 

The  importance  of  color  is  obviously  task  dependent,  and  may 
be  oveiplayed.  In  this  regard  it  is  interesting  to  note  that  Murphy 
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Figure  1.  Massimino’s  results  showing  decreasing  completion  time  of 
block  insertion  as  (a)  visual  angle  and  (b)  frame-rate  increase.  Force 
feedback  and  no  force  feedback  conditions  are  compared.  Bars 
indicate  standard  error  of  the  mean.  [From  Massimino,  ref.  2] 


et  al.[3]  found  that  experienced  dermatologists  could  diagnose 
skin  lesions  as  well  over  black-and-white  video  as  over  color 
video. 

The  tradeoff  among  frame  rate,  resolution,  and 
grayscale  for  bandlimited  communication 

When  signal  bandwidth  is  limited,  as  is  often  the  case  for  deep 
space  radio  or  underwater  sound  communication,  it  may  be 
important  to  ask  how  best  to  trade  off  the  three  variables  of  frame 
rate  (frames  per  second),  resolution  (pixels  per  frame),  and 
grayscale  (bits  per  pixel),  the  product  of  which  is  bandwidth  (bits 
per  second). 

These  tradeoffs  were  studied  by  Ranadive  [4]  in  the  context  of 
master-slave  manipulation.  Experimental  subjects  were  asked  to 
perform  two  simple  assembly  tasks  using  a  video  display  as  their 
only  feedback  while  using  a  seven-degree-of-freedom  remote 
manipulator.  The  video  display  was  systematically  degraded  with 
a  special  electronic  device  that  allowed  the  frame  rate  to  be 
adjusted  to  28,  16,  8,  or  4  frames  per  second,  resolution  to  be 
adjusted  to  128,  64,  32,  or  16  pixels  linear  resolution,  and 
grayscale  to  be  adjusted  to  4,  3,  2,  or  1  bits  per  pixel  (i.e.,  16,  8, 
4,  or  2  levels  of  CRT  intensity).  The  data-collection  runs  were 
ordered  so  that  two  of  the  three  video  variables  were  kept 
constant  while  the  third  was  varied  randomly  among  the  levels 
for  that  variable. 

Figure  2  shows  the  results.  On  the  top  row  are  shown  the 
performance  effects  of  frame  rate,  resolution,  and  grayscale  while 
the  other  variables  are  held  constant.  Note  that  for  frame  rates 
beyond  16  frames  per  second  improvement  depends  on 
resolution  and  grayscale;  performance  improves  smoothly  for 
increases  in  resolution;  for  grayscale  there  is  no  improvement 
beyond  2  bits  if  the  frame  rate  is  high  enough.  On  the  bottom 
row,  constant-level-of-performance  tradeoffs  (in  this  case  using 
the  take-off-nut  task  only)  are  shown  for  each  of  the  three  pairs 
of  video  variables.  These  iso-performance  curves  (solid  lines)  are 
compared  to  iso-transmission  lines,  i.e.,  combinations  of  the  two 


parameters  which  produce  constant  bits  per  second.  It  is  seen  that 
there  is  a  remarkable  correspondence.  This  means  that,  for  this 
experiment,  and  within  the  range  of  video  variables  employed, 
human-and-machine  performance  corresponds  roughly  to  bits  per 
second  of  the  display,  regardless  of  the  particular  combination  of 
frame  rate,  resolution,  and  grayscale. 

If  a  limited-bandwidth  channel  must  be  used  as  the  means  for 
communication  between  human  operators  and  teleoperators,  it 
seems  reasonable  to  allocate  these  fixed  channels  as  required  by 
the  task  at  each  moment.  That  is,  frame  rate,  resolution,  and 
grayscale  would  not  each  have  fixed  bandwidth  allocations; 
rather,  provision  would  be  made  to  trade  off  among  these  as 
needed,  keeping  their  product  as  close  as  possible  to  the 
maximum. 

Deghuee  [5]  used  an  experimental  computer-based  aiding  device 
which  allowed  the  operator  to  make  this  three-way  adjustment  in 
situ,  i.e.,  he  could  adjust  the  F-R-G  tradeoff  himself  while 
performing  a  master-slave  manipulation  task  of  the  type 
performed  in  Ranadive’s  experiments.  As  might  be  expected,  by 
use  of  the  tradeoff  control  performance  was  significantly  better  (p 
<  0.05). 

Depth  cues:  stereo  and  other 

Gaining  a  sense  of  depth  is  the  most  difficult  visual  problem 
when  viewing  through  closed  circuit  video.  To  recreate  the 
stereo  sense  of  depth  obtained  when  viewing  a  real  object  with 
two  eyes,  different  2D  images  must  be  obtained  from  two 
horizontally  separated  viewpoints,  then  presented  to  the 
corresponding  left  and  right  retinas.  The  brain  recreates  the  3D 
information  from  a  variety  of  cues,  of  which  binocular  disparity 
is  but  one.  In  direct  viewing,  other  strong  3D  cues  are 
accommodation,  shadows,  prior  knowledge  of  relative  size  and 
of  what  object  is  behind  what  other  object,  and  motion  parallax 
(i.e.,  the  ability  to  move  the  head  from  side  to  side  and  gain  a 
different  viewpoint).  None  of  the  latter  cues  requires  two  eyes.  In 
televiewing  accommodation  is  not  available,  and  motion  parallax 
is  available  only  with  head-mounted  or  other  head-position¬ 
measuring  display  techniques). 

For  teleoperation  the  images  can  be  obtained  by  two  separate 
video  cameras,  or  by  a  single  camera  outfitted  with  two  optical 
paths  sharing  the  video  field  in  time  or  space,  or  by  a  geometric 
model  run  in  a  computer.  Presentation  of  the  images  can  be  by 
means  of  two  separate  optical  paths  (one  to  each  eye)  or  by  a 
single  display  which  provides  two  images  in  parallel.  The  latter 
can  be  separated  for  each  eye  by  color  filtering  (wearing  red  and 
green  glasses)  or  by  temporal  shuttering  (alternate  presentation  to 
each  eye  of  each  corresponding  image).  Image  transmission  must 
maintain  proper  size,  shape,  brightness,  and  color  (if  color  is 
displayed  as  such  and  not  used  for  binocular  channel  separation). 
Many  studies  confirm  a  significant  reduction  in  task  performance 
time  by  use  of  stereo  [6]  [7]. 

Stereopsis  is  not  always  practical;  even  when  it  is,  depth 
perception  continues  to  be  a  major  reason  why  performance  of 
direct  manipulation  is  noL  matched  by  that  of  telemanipulation. 
Winey  [8]  evaluated  three  means  to  provide  depth  cues  on  a  video 
or  computer  display:  front  plus  side  views  (orthographic 
projection),  artificially  generated  shadows  projected  on  an 
imaginary  horizontal  floor,  and  an  analog  proximity  indication  of 
the  distance  between  the  gripper  and  the  manipulated  object.  The 
operator’s  task  was  to  operate  a  computer-simulated  six-DOF 
manipulator  so  as  to  reach  out  and  grasp  a  simulated  sphere  or 
block,  which  was  stationary  in  some  cases  and  moving  in  others. 
On  both  stationary  and  moving  tasks,  front  and  side  orthographic 
projections  showed  the  best  performance.  However,  all  the 
subjects  felt  the  shadow  gave  them  the  best  perception  of  the 
object’s  position  in  the  environment.  Winey  suggested  that  the 
front  and  side  views  be  combined  with  the  shadow,  where  the 
shadow  is  used  to  provide  an  intuitive  perception,  while  the  side 
and  front  views  provide  the  detail. 

Stereopsis  is  not  the  only  depth  cue  for  video.  Kim  et  al.  [9] 
showed  that  superposing  in  the  video  display  some  computer¬ 
generated  perspective  grid  lines,  with  equi-depth  reference  lines 
drawn  from  the  reference  grid  to  important  objects,  makes  it  easy 
for  the  observer  to  comprehend  the  relative  depth  of  the  objects. 
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Figure  2.  Ranadive’s  results  for  frame-rate,  resolution,  grayscale 
tradeoff.  Top  row,  each  plot  shows  performance  as  one  of  the  three 
variables  is  improved,  the  other  two  kept  constant.  Bottom  row,  each 
plot  compares  constant  (iso)  performance  and  constant  (iso) 
information  transmission  as  a  function  of  two  of  the  three  variables 
simultaneously,  the  other  held  constant.  TON  is  take-off-nut  task. 
[After  Ranadive,  ref.  4] 


Superposing  other  essential  graphics  onto  the  video  picture, 
much  as  the  aircraft  pilot’s  “head-up”  display  is  superposed  onto 
the  windscreen,  is  a  convenient  way  to  save  the  human  operator 
from  having  to  keep  accommodating  from  computer  displays  to 
video  and  back  again.Computer-generated  components  of  a 
display  can  be  made  to  look  realistic  and  then  become  “virtual 
displays.” 

Teleproprioception  and  Kinesthesis 

Proprioception  is  literally  “awareness  of  self’.  Gravity  provides 
a  strong  vestibular  cue  and  a  directional  static  loading  on  one’s 
own  body,  so  that  muscle  reflexes  are  driven  automatically  to 
maintain  posture  and  body-position  awareness.  With  a 
teleoperator  these  cues  are  normally  missing,  or  at  least  there  is  a 
severe  problem  of  establishing  anything  approaching  the  tight 
coupling  between  proprioceptive  sensors  and  brain.  Kinesthesis 
is  literally  sense  of  motion.  Kinesthesis  and  proprioception  are 
terms  often  used  together  by  psychologists,  at  least  in  part 
because  the  same  receptors  in  the  human  body’s  muscles  and 
tendons  mediate  both.  For  that  reason  we  lump  them  together  in 
our  discussion. 

Telekinesthesis  and  teleproprioception  are  particularly  critical 
because,  as  telemanipulation  experience  has  shown,  it  is  very 
easy  for  the  operator  to  lose  track  of  the  relative  position  and 
orientation  of  the  remote  arms  and  hands  and  how  fast  they  are 
moving  in  what  directions.  This  is  particularly  aggravated  by 
one’s  having  to  observe  the  remote  manipulation  through  video 
without  peripheral  vision  or  very  good  depth  perception,  or  by 
not  having  master-slave  position  correspondence,  i.e.,  when  a 
joystick  is  used.  Potential  remedies  are  multiple  views,  wide  field 
of  view  from  a  vantage  point  which  includes  the  arm  base,  and 
computer-generated  images  of  various  kinds  (the  latter  will  be 
discussed  further  below).  Providing  better  sense  of  depth  is 
critical  to  telemanipulation  anywhere. 

Correspondence  of  display  and  control; 
anthropomorphic  design 

Closely  related  to  teleproprioception  and  telekinesthesis  is  the 
question  of  whether  there  should  be  a  one-to-one  correspondence 
between  displays  (of  whatever  sensory  modality)  and  the  control 


devices  upon  which  the  operator  directly  acts  with  her  hands  (or 
feet  or  body).  A  fundamental  principle  of  human-factors 
engineering  is  to  design  the  controls  and  displays,  insofar  as 
possible,  so  that  a  control  action  and  a  resulting  change  in  the 
display  are  in  the  same  relative  location  with  respect  to  the  other 
controls  and  displayed  variables,  and  move  in  the  same  direction. 
One  way  to  ensure  that  this  condition  obtains  is  to  design  the 
teleoperator  anthropomorphically  —  having  the  same  form  as  the 
human  operator  —  and  then  arrange  at  least  the  analogic  controls 
(e.g.,  a  master  arm  or  joysticks),  and  perhaps  even  some  of  the 
symbolic  controls,  with  geometric  correspondence  to  what  they 
control  on  the  teleoperator.  Of  course  a  conventional  identical 
master-slave  system  fulfills  this  condition. 

The  operator  often  must  orient  the  teleoperator  arm  at  a  fixed 
location  and  orientation,  for  example  a  peg  adjacent  to  and 
aligned  with  a  hole.  How  well  can  this  be  done  using  a  2D  video 
display?  Yoerger  [10]  tested  the  operator’s  ability,  viewing  both 
directly  and  through  video,  to  orient  the  slave  hand  normal  to  a 
plane  when  the  plane  was  at  different  angles  to  the  view 
direction.  He  found  that  there  were  significant  orientation  errors 
as  a  function  of  plane’s  orientation  relative  to  view  direction,  and 
that  for  both  direct  and  video  viewing  subjects  consistently 
underestimated  the  angle  between  the  view  and  the  plane.  A  45° 
angle  between  the  view  direction  and  the  plane  was  found  to  be 
best. 

If  there  is  a  90°  or  greater  rotation  between  hand  movements  and 
(lagged)  displayed  movements  of  the  controlled  object, 
performance  in  tracking  and  manipulation  deteriorates.  Bernotat 
[11]  showed  that  adding  an  indicator  of  hand  position  to  the 
rotated  display  improves  tracking  performance,  but  that  the 
performance  reverts  when  the  hand-position  cue  is  removed^ 
Cunningham  and  Pavel  [12]  used  an  even  more  difficult  108° 
rotation  of  the  display  in  a  discrete  aiming  task,  adding  a  novel 
“wind  indicator”  to  the  display  which  provided  a  virtual  causation 
of  the  rotational  bias.  Subjects  were  instructed  to  oppose  the 
virtual  force  represented  by  the  indicator.  This  enhancement 
reduced  aiming  error  by  70  percent  in  the  first  10  minutes  of 
practice,  and  aiming  error  did  not  rise  after  removal  of  the  cue. 
This  suggests  that  biases  caused  by  display-control  rotations  can 
be  overcome  with  proper  cueing. 
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FORCE  AND  TOUCH  SENSING  AND  DISPLAY 
The  nature  of  force  feedback 

Resolved  force  sensing  is  what  the  human  body’s  joint,  muscle, 
and  tendon  receptors  do  to  determine  the  net  reaction  force  and 
torque  acting  on  the  hand,  i.e.,  the  vector  resultant  of  all  the 
component  forces  and  torques  of  the  hand  acting  on  the 
environment.  Various  limbs  can  perform  this  measurement  over  a 
wide  dynamic  range  and  with  a  just-noticeable  difference  of  6-8 
percent  in  the  2-10  newton  range.  In  force-reflecting  master- 
slave  systems  such  resolved  forces  are  measured  at  the  slave  end 
either  by  strain-gauge  bridges  in  the  wrist  (so-called  wrist-force 
sensors),  by  position  sensors  in  both  master  and  slave  (which, 
when  compared,  indicate  the  relative  deflection  in  six  DOF, 
which  in  the  static  case  corresponds  to  force),  or  by  electrical 
motor  current  or  hydraulic  actuator  pressure  differentials. 

Display  of  feedback  to  the  operator  can  be  straightforward  in 
principle;  in  force-reflecting  master-slave  systems  the  measured 
force  signals  drive  motors  on  the  master  arm  which  push  back  on 
the  hand  of  the  operator  with  the  same  forces  and  torques  with 
which  the  slave  pushes  on  the  environment.  This  might  work 
perfectly  in  an  ideal  world  where  such  slave-back-to-master  force 
tracking  is  perfect,  and  the  master  and  slave  arms  impose  no 
mass,  compliance,  viscosity  or  static  friction  characteristics  of 
their  own.  But  not  only  does  reality  not  conform  to  this  dream;  it 
can  also  be  said  that  we  hardly  understand  what  are  the 
deleterious  effects  of  these  spurious  mechanical  characteristics  in 
masking  the  sensor}'  information  that  is  sought  by  the 
telemanipulation  operator,  or  how  to  minimize  these  effects. 
Corker  and  Bejczy  [13]  showed  that  master  and  slave  need  not 
have  the  same  kinematics  if  force  reflection  is  to  be  used;  a 
computer  can  transform  coordinates  ans  still  produce  proper  force 
feedback.  It  has  also  been  shown  that  force  reflection  can  be 
applied  to  a  rate-control  joystick  [14]. 

Force  feedback  masking  and  teleoperation 
performance 

There  are  several  factors  in  master-slave  teleoperation  which 
contribute  to  insensitivity  to  contact  or  other  forces.  These  can 
result  in  instability  because  the  operator  may  not  feel  the  forces 
imposed  on  the  slave  by  the  environment  and  will  keep  moving 
the  master  when  force  feedback  should  signal  him  to  stop  or  to 
reverse  direction. 

There  is  effective  masking  of  forces  felt  by  the  operator  because 
the  mechanism  of  the  force-reflecting  hand  controller  may  have 
significant  coulomb  friction  (“stiction”)  force  Fc  viscous  friction 
force  Fv  (Fc  and  Fv  would  not  occur  at  the  same  time),  inertial 
force  Fj,  and  gravity  force  Fg  between  the  force  feedback 
actuators/sensors  and  the  handgrip,  all  of  which  can  cancel  or, 
because  they  might  be  larger  than  feedback  forces  from  the  slave, 
confuse  the  operator  as  to  what  is  their  source.  These  masking 
forces  add  to  the  operator’s  own  sensory  threshold  Fs  for  force 
detection.  This  effect  is  multiplied  by  whatever  ratio  R  obtains  for 
force  feedback  transferred  to  the  master  relative  to  forces  applied 
to  the  slave  by  the  environment.  Combining  these  factors  results 
in  a  net  force  threshold  Fj: 

Fy  =  R  (Fc  +  Fv  +  F[  +  Fg  +  Fs). 

Fc  and  Fs  are  usually  the  major  culprits. 

Force  reflection  was  inherent  in  the  original  direct  mechanical 
cable-connected  master-slave  manipulators  of  Goertz  [15],  and 
was  designed  into  the  early  electrical  and  hydraulic  master-slave 
systems  as  noted  earlier.  Numerous  studies  have  been  performed 
over  the  years  to  evaluate  whether,  and  under  what 
circumstances,  force  feedback  helps  performance 
[16][17][18][19].  Massimino  et  al.[2]  found  that  force  feedback 
made  a  consistently  significant  difference  and  cut  task-completion 
times  almost  in  half. 

Computer-graphic  display  of  force-torque  information 

Force-torque  information  is  easy  to  provide  visually,  though 


there  is  a  question  whether  this  is  the  best  sensory  mode  to 
receive  force  feedback.  Figure  3  shows  a  computer-graphic 
force-torque  display  developed  by  Bejczy  [20].  The  bars  at  the 
center  provide  a  pseudo-perspective  view  of  hand  coordinates  X, 
Y,  and  Z.  The  diagonal  bar  represents  the  Fx  translational  forces 
(in  and  out  of  screen).  Bars  at  upper,  right,  and  lower  edges 
represent  the  moments  around  the  X,  Y,  and  Z  axes, 
respectively.  The  two  vertical  bars  on  the  left  show  finger 
opening  and  clamping  forces. 


Figure  3.  Bejczy’s  force-torque  display.  [Courtesy  of  Jet  Propulsion 
Laboratory,  California  Institute  of  Technology.] 


Sharing  control  between  teleoperation  and  computer 
autonomy  to  achieve  force  control 

Earlier  some  mention  was  made  of  trading  and  sharing  of  control 
between  human  and  computer.  Hayati  and  Venkataraman  [21] 
developed  an  architecture  for  a  flexible  telerobot  which  shares 
control  by  a  human  operator  using  a  full  multi-degree-of-freedom 
position  master  and  a  semi-autonomous  computer.  Their  scheme 
allows  command  signal  vectors  from  both  on-line  teleoperator 
and  computer  to  be  weighted  and  added,  where  the  weighting 
coefficients  depend  on  factors  such  as  time  delay  and  whether  the 
telerobot  is  operating  in  “free  motion,  guarded  motion,  fine 
motion,  free  force  application,  guarded  force  application  or  fine 
force  application.”  For  example,  when  in  free  motion, 
teleoperator  inputs  do  not  affect  system  stability,  and  thus  can  be 
allowed  to  dominate.  However,  in  any  force  application  or  in 
guarded  motion  (i.e.,  where  a  contact  surface  is  nearby)  contact- 
force  instability  is  likely,  and  thus  teleoperator  weighting  is 
reduced.  During  fine  motions  teleoperation  weighting  is  allowed 
along  motion  directions,  but  suppressed  along  other  direction 
which  may  involve  contact  forces.  The  position  and  force 
feedback  vector  from  the  telerobot  is  similarly  subjected  to 
weighting  matrices,  where  the  coefficients  are  determined  by 
criteria  of  stability  and  the  separation  of  error  contribution  from 
each  separate  input  agent.  Figure  4  illustrates  the  weighting 
scheme.  The  weighting  matrices  are  defined  relative  to  the 
(preplanned)  task  space;  each  of  six  degrees  of  freedom  can  be 
set  in  one  of  ten  modes  independently,  resulting  in  10^ 
combinations. 

Comparison  of  task  sharing  with  alternative  control 
and  force  feedback  modes 

Hannaford  et  al.  [22]  performed  an  experimental  evaluation  of 
several  command  and  sensory  feedback  modes  including  task 
sharing  as  just  described  above  (one  of  the  many  combinations  of 
control  modes),  using  the  Jet  Propulsion  Lab  /  Salisbury 
positional  master  arm  and  a  Westinghouse  Puma  560  robot  arm 
serving  as  a  slave.  The  five  modes  were  (1)  position  control  with 
visual  feedback  only,  (2)  position  control  with  visual  display  of 
position  and  force  (as  described  above  in  the  subsection  on 
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HUMAN 


Figure  4.  Weighting  scheme  of  Hayati  and  Venkataraman  (simplified 
by  author)  for  sharing  control  between  computer  and  human,  inputs 
from  both  computer  C  and  human  H  are  transformed  by  weighted 
control  matrices  for  both  the  movement  M  and  force  F,  then 
appropriately  combined  and  subjected  to  Jacobian  transformations  T 
as  shown.  The  reverse  is  done  to  provide  feedback  to  the  computer 
and  human.  [After  Hayati  and  Venkataraman,  ref.  21] 


graphical  force  display),  (3)  bilateral  force  feedback  to  the  master 
ami  (which  they  termed  “kinesthetic  control”),  (4)  shared  control 
(based  on  the  scheme  just  described  above),  and  (5)  direct  bare¬ 
handed  control.  In  this  case  of  “shared  control”  the  operator’s 
force  commands  were  added  to  those  of  automatic  force 
accommodation  for  orientation  axes  and  fine  position  control, 
while  conventional  force  feedback  to  the  human  was  used  for  free 
translation.  They  used  four  tasks  in  their  experiments:  attaching 
and  detaching  blocks  covered  with  velcro,  a  matrix  of  peg-in-hole 
tasks  with  different  size  pegs  and  different  size  chamfers,  mating 
and  unmating  several  standard  electrical  connectors,  and 
unmating,  mating,  and  locking  an  electrical  bayonet  connector. 
Results  are  shown  in  Figure  5.  In  Figure  5,  for  two  of  the  tasks 
the  shared  control  proved  significantly  better  in  completion  time 
than  kinesthetic  control,  and  the  force  levels  (actually  a  sum  of 
squared  force  integrated  over  time)  even  approached  those  for 
bare-handed  (“manual”)  manipulation.  Results  from  these 
experiments  underscore  that  shared  teleoperation  and  computer 
control  is  promising,  but  the  number  of  arrangements  yet  to  be 
evaluated  is  staggering. 

Impedance  control 

Impedance  is  normally  defined  as  the  relation  between  applied 
force  F  and  velocity  V.  For  a  linear  system  impedance  Z  is 
commonly  defined  as 

Z=F I  V  =  Ms  +  B  +  K/s 

where  M  is  mass,  B  is  viscous  damping,  K  is  stiffness,  and  s  is 
the  Laplace  argument  or  time-derivative  equivalent  in  the  time 
domain.  This  implies  a  common  null  point  for  all  terms. 

The  variable  stiffness  (campliance)  aspect  of  impedance  control 
for  a  telerobot  may  be  thought  of  in  terms  of  a  hypothetical 
“compliance  frame”  (Figure  6)  attached  to  the  end  point  of  the 
teleoperator  arm  (master  or  slave  side,  whichever  one  chooses  to 
consider).  In  this  case  it  becomes  clear  that  the  compliance  null 
point  can  be  moved  in  position,  while  the  compliance  and/or 
viscosity  parameters  (mass  is  usually  neglected)  can  be  adjusted 
independently.  Thus  a  relatively  constant  force  may  be  applied  to 
an  object  in  spite  of  small  arm  motions  relative  to  it  by 
commanding  the  end  point  compliance  to  be  soft  and  adding  a 
large  equivalent  position  bias  in  the  desired  force  direction  . 
Alternatively,  a  relatively  constant  position  may  be  imposed  on  an 
object  by  a  stiff  spring  (what  we  usually  think  of  as  position 
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Figure  5.  Hannaford’s  time  and  force  results  for  task  sharing  with 
alternative  control  and  force  feedback  modes,  (a)  Averaged 
completion  time  and  force  performance  for  peg-in-hole  task,  (b) 
Averaged  completion  time  and  force  performance  for  electrical 
connector  attachment  task.  [From  Hannaford  et  al.  ,  ref.  22] 


control).  In  fact,  subject  to  constraints  of  stability  and  actuator 
limits,  any  desired  end  point  impedance  to  motion  (or  admittance 
of  forces)  may  thus  be  programmed  to  mimic  the  desired 
compliance,  viscosity,  and  mass  parameters  of  the  end  point. 
These  parameters  may  even  be  different  in  different  directions,  or 
change  with  time  —  which  seems  to  be  what  we  do  with  our  own 
limbs  in  catching  balls,  threading  needles,  and  other  ordinary 
manipulation  tasks  [23]  [24], 

The  best  impedance  for  a  master-slave  manipulator 

There  is  a  diversity  of  opinion  about  what  constitutes  the  “best” 
impedance  for  a  master-slave  teleoperator.  One  argument  [25]  is 
that  an  ideal  teleoperator  is  one  that  is  transparent,  i.e.,  the 
equivalent  of  an  infinitely  stiff  and  weightless  mechanism 


Figure  6.  Hypothetical  end-point  compliance  frame  (equivalent  to 
impedance  felt  by  externally  imposed  forces).  [From  Sheridan,  ref.  1] 
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between  the  end  effector  of  the  master  arm  and  the  operator’s 
hand  assembly  of  the  master  arm.  Vertut  and  Coiffet  [18]  have 
suggested  instead  that  operators  get  tired  when  holding  their  arms 
in  fixed  and  awkward  positions  and/or  applying  constant  forces 
(as  master-slave  systems  often  require),  and  the  author’s 
experience  confirms  this.  Bejczy  and  Handylykken  [19]  report 
that  there  seem  to  be  different  best  combinations  of  force- 
feedback  gain  (from  slave  to  master)  and  feedforward  gain  (from 
master  to  slave)  for  different  tasks. 

Providing  for  the  operator  to  adjust  the  impedance  of  the  master 
and/or  the  slave  may  be  a  promising  way  of  making  a  master- 
slave  teleoperator  more  versatile  than  if  the  compliance-viscosity- 
inertance  parameters  remained  fixed.  A  carpenter  may  carry  and 
use  within  one  task  several  different  hammers,  and  a  golfer  many 
clubs,  because  each  different  tool  provides  an  impedance 
characteristic  appropriate  for  particular  task  conditions  which  are 
expected.  Carrying  many  teleoperators  into  space,  for  example, 
may  be  avoided  by  making  the  impedance  adjustable  between 
slave  and  task  and/or  between  human  and  master. 

Should  the  impedance  of  the  human  arm  serve  as  a  model?  The 
human  arm  has  amazing  capability.  Also,  as  we  shall  see  below, 
the  impedance  seen  at  the  master  port  of  a  master-slave 
manipulator  is  to  some  extent  dependent  on  the  human  operator’s 
arm  impedance.  As  Hogan  [23]  reports,  much  evidence  supports 
the  notion  that  the  human  arm  can  be  modeled  as  having  passive 
impedance.  This  model  does  not  imply  that  the  human  arm  is 
passive.  However,  the  active  part  of  the  human  arm  dynamics 
can  be  considered  as  a  state-independent  force  source  that,  at  least 
in  the  linear  case,  does  not  affect  the  system’s  stability.  At  the 
same  time,  there  is  evidence  that  the  human  arm’s  impedance 
varies  over  a  wide  range,  though  the  rapidity  of  adaptation  may 
be  limited.  Perhaps  the  lesson  from  traditional  manual  control, 
namely  the  McRuer  [26]  crossover  model  could  be  applied  to  the 
human  arm  and  manipulation  task,  namely  that  the  combination 
of  the  arm  and  the  task  are  much  less  variant  than  either  by  itself, 
and  therefore  the  simplest  and  probably  most  useful  model 
should  be  a  model  of  the  combination  . 

The  skin  senses  of  touch 

Touch  is  a  term  used  sloppily  to  refer  to  various  forms  of  force 
sensing,  but  more  precisely  to  refer  to  the  sense  of  differential 
forces  (or,  equivalently,  displacements)  on  the  skin  in  time  and  in 
space,  both  normal  and  tangential  to  the  skin  surface.  (The  skin 
can  sense  other  stimuli,  of  course,  such  as  heat,  cold,  pain,  etc.) 
The  skin  is  a  poor  sensor  of  absolute  magnitude  of  force,  and  it 
adapts  quickly. 

Five  types  of  nerve  fibers  mediate  touch  [27]  [28]: 

(1)  Slowly  adapting  type  I  fibers  terminate  at  the  base  of 
the  epidermis  in  Merkel  cells,  are  distributed  densely  (one  per 
square  mm),  respond  to  temporal  stimuli  in  the  range  1—1 00  Hz, 
have  a  sensitive  range  of  0.03-3.0  mm  skin  indentation,  and  are 
acutely  responsive  to  edges  and  regions  of  curvature. 

(2)  Rapidly  adapting  type  I  fibers  terminate  at  the  base  of 
the  epidermis  in  Meissner  corpuscles,  are  also  distributed  densely 
(one  per  square  mm),  have  a  bandpass  of  2-200  Hz  with  peak 
sensitivity  near  50  Flz,  have  a  sensitive  range  of  0.001-1  mm 
indentation,  and  are  less  spatially  and  temporally  acute  than  type 

1  above. 

(3)  Slowly  adapting  type  II  fibers  terminate  in  deep 
Ruffini  structures,  are  less  densely  distributed  (10  per  square 
cm),  have  low-pass  temporal  response  (0-10  Hz),  and  are 
primarily  responsive  to  horizontal  skin  stretch. 

(4)  Rapidly  adapting  type  II  fibers  terminate  in  deep 
Pacinian  corpuscles,  have  a  temporal  bandpass  of  20-1000  Hz 
with  peak  sensitivity  at  30CM-00  Hz,  and  are  extremely  sensitive 
(100-1000  angstroms  peak  to  peak)  to  skin  amplitude  vibrations 
generally,  much  less  at  the  receptor  ending. 

(5)  Hair  follicle  receptors  should  also  be  added,  which 
respond  to  light  axial  or  bending  forces,  with  spatial 
discrimination  from  0. 1-3  cm. 

Touch  sensing  contexts  and  quantitative  theory 

Touch  sensing  and  perception  may  be  considered  in  three 
different  sensorimotor  contexts: 


(1)  Forces  are  imposed  on  the  skin  by  the  environment 
without  any  overt  intentional  movements  made  relative  to  the 
source  of  those  forces.  This  may  be  called  passive  or  non-haptic 
touch. 

(2)  The  movements  of  the  touch  sensor  are  made 
voluntarily  in  order  to  explore  some  portion  of  the  mechanical 
environment,  and  to  achieve  touch  identification  of  one  or  more 
objects  and  their  positions  and  orientations.  This  may  be  called 
pure  active  or  haptic  touch,  where  kinestheic  sensation  may  be 
correlated  with  cutaneous  sensation  to  infer  patterns  in  time  and 
space. 

(3)  Touch  sensing  is  done  as  an  integral  part  of  actively 
manipulating  with  the  hand  or  moving  the  body  to  perform  some 
task  not  primarily  one  of  touch  sensing. 

Context  1  may  seem  to  be  the  equivalent  of  visual  and 
electromagnetic  image  recognition  and  understanding.  A  great 
deal  of  quantitative  theory  has  been  applied  to  this  problem  for 
applications  in  robot  vision,  space  photography  of  the  earth  and 
the  heavens,  biomedical  imaging,  etc.  Unfortunately,  cutaneous 
patterns  do  not  seem  to  be  perceived  with  enough  resolution  and 
memory  to  make  much  of  this  available  theory  applicable. 

Context  2  is  now  seeing  some  research  in  a  telerobotic  context, 
which  may  also  offer  a  way  into  context  3. 

Context  3  has  produced  little  research  that  is  coherent,  not 
because  it  is  not  recognized  as  being  an  important  problem,  but 
because  it  is  so  difficult,  and  because  the  modes  of  “touching  in 
the  precess  of  doing”  are  so  many  and  varied.  To  this  writer’s 
knowledge  no  generally  accepted  theoretical  or  experimental 
paradigms  have  emerged. 

Though  “labors  in  these  lower  vineyards  of  the  sensory  domain” 
[27]  have  not  provided  neat  quantitative  paradigms,  the  literature 
on  touch  perception  is  extensive.  An  excellent  review  is  that 
provided  by  Loomis  and  Lederman  [29],  which  includes 
determination  of  absolute  and  differential  thresholds  as  a  function 
of  force  magnitude  and  direction  relative  to  the  skin,  time, 
frequency,  body  locus,  two-point  separation,  stimulus  size  and 
shape  (including  texture),  recognition  among  previously  learned 
patterns,  and  the  effects  of  masking  on  all  of  these.When 
discrimination  and  recognition  are  sufficiently  large  that  multiple 
fingers  and  reshaping  of  the  hand  are  required,  that  is  called 
stereognosis. 

Touch  sensing  and  display  devices 

There  are  now  a  few  devices  for  artificial  teletouch  sensing.  Most 
of  these  have  much  coarser  spatial  resolution  than  the  skin,  such 
as  the  very  first  touch  sensors  for  telerobots  (which  consisted  of 
a  few  microswitches  placed  at  gripping  surfaces  or  where 
obstacle  collisions  might  occur).  Various  devices  have  now  been 
marketed  (e.g,  by  the  Lord  Corporation)  which  are  relatively 
coarse  arrays  of  magnetic,  resistive,  capacitive,  or  optical 
continuous  force/displacement  elements.  Harmon’s  [30] [31] 
reviews  of  the  state  of  tactile  sensors  for  robots  are  surprisingly 
current. 

The  most  difficult  problem  for  teletouch  is  not  sensing  but 
display.  How  should  artificially  sensed  pressure  patterns  be 
displayed  to  the  human  operator?  One  would  like  to  display  such 
information  to  the  skin  on  the  same  hand  that  is  operating  the 
joystick  or  master  arm  which  guides  the  remote  manipulator.  This 
has  not  been  achieved  successfully,  the  major  reason  being  that 
the  skin  receptors  are  masked  by  the  forces  of  gripping  the  handle 
as  well  as  the  reaction  forces  of  inertia,  friction,  and  spring¬ 
centering  (if  any)  of  the  master.  An  option  is  to  display  to  the 
skin  at  some  other  location  than  at  the  handle-gripping  surfaces. 
Much  of  the  early  research  in  tactile  displays  was  directed  toward 
aiding  the  blind.  Included  have  been  arrays  of  electromagnet 
vibrators,  piezoelectric  bimorphs  (up  to  64x64  such  bimorph 
vibrators  have  been  packaged  into  a  7x7-inch  array).  More 
recently  alloys  such  as  TiNi  which  change  their  length  when 
heated  have  shown  promise. 

Most  of  the  success  in  teletouch  has  been  achieved  by  displaying 
remote  tactile  information  to  the  eyes  using  a  computer-graphic 
display  [32] [33],  The  above  problems  for  tactile  teleoperation  are 
in  spite  of  the  fact  that  without  vision  one  can  easily  track  a 
randomly  moving  tactile  stimulus  almost  as  well  as  a  visual  one. 
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This  was  shown  by  Weissenberger  and  Sheridan  [34],  who  used 
a  handle  lightly  gripped  between  the  thumb  and  the  index  finger 
to  equalize  the  pressure,  and  by  Jagacinski  et  al.  [35]  with  a 
similar  display. 

TELEPRESENCE  AND  VIRTUAL  PRESENCE 
Telepresence 

Telepresence  is  commonly  claimed  to  be  important  for  direct 
manual  telemanipulation.  It  has  yet  to  be  shown  how  important  is 
the  sense  of  “feeling  present”  per  se  as  compared  to  simply 
having  high  resolution,  a  wide  field  of  view,  and  other  attributes 
of  good  sensory  feedback.  Further,  although  telepresence  is 
usually  identified  with  direct  manual  teleoperation,  it  may  be  just 
as  important  to  be  able  to  “feel  present”  when  supervising  a  semi- 
autonomous  telerobot. 

In  addition  to  visual  telepresence,  there  are  auditory  telepresence 
(binaural  localization  and  spectral  correspondence  to  the  real 
world),  resolved  force  (muscle  force)  telepresence,  tactile  (skin 
sense)  telepresence,  and  vestibular  telepresence  (achievable 
through  a  motion  platform  driven  by  the  same  disturbances  the 
operator  would  be  subjected  to  were  he  at  the  remote,  or  virtual, 
location). 

Many  of  the  mechanical-force  or  other  disturbances  which  might 
contribute  to  one’s  sense  of  telepresence  are  the  very  things  one 
seeks  to  avoid.  For  example,  one  often  seeks  to  avoid  vibration 
or  sudden  unexpected  mechanical  forces  which  interfere  with 
visual-motor  skills.  One  seeks  to  avoid  extremes  of  temperature 
and  pressure,  explosions,  and  other  hazards  which  might 
contribute  further  to  telepresence  but  which  may  be  the  very 
reasons  for  teleoperation  in  the  first  place.  Therefore,  “full” 
telepresence  is  a  questionable  goal  in  many  situations. 

Tachi’s  experiments  in  telepresence  using  a  helmet- 
mounted  display 

Tachi  et  al.  [36]  developed  and  evaluated  the  hardware 
components  to  implement  teleoperator  telepresence  (their  term  is 
tele-existence).  At  last  report  their  head-mounted  display  was 
binocular,  with  4-inch  color  liquid-crystal  displays  (320x220 
pixels)  and  eyeglasses  used  to  achieve  close  focus  on  the  LCDs. 
The  system  was  helmet  mounted  and  weighed  1 .7  Kg  in  all.  An 
earlier  system  [37]  used  3-inch  CRTs  in  a  5-  Kg  assembly 
mounted  in  a  five-DOF  head-following  servomechanism.  With 
both  systems  they  achieved  good  subjective  report  of 
telepresence.  The  second  component  of  Tachi’s  system  was  a 
six-DOF  electromagnetic  sensor  similar  to  the  Polhemus  sensor 
described  above.  It  had  three  orthogonal  10-KH  fields,  and  he 
achieved  accuracies  of  2.5  mm  in  translation  and  0.5°  in  rotation 
within  a  1.5-m3  workspace.  This  drove  a  specially  built  seven- 
DOF  slave  robot  which  was  purposely  anthropomorphic  in 
design  to  permit  easy  “tele-identification”  with  it.  It  also  had  a 
one-  DOF  torso  rotation  for  slaving  to  the  operator’s  waist  twist. 
Figure  7  shows  the  operator  with  helmet-mounted  display  and  the 
anthropomorphic  teleoperator. 

The  third  component  was  a  three-wheeled  remotely  driven  vehicle 
on  which  a  pan-tilt-stereo  video  system  was  mounted  [38].  The 
vehicle  was  driven  by  a  manual  joystick  and  the  video  driven  by  a 
head-mounted  display,  which  also,  of  course,  received  the  video 
signals.  Auditory  signals  from  microphones  on  either  side  of  the 
vehicle  were  fed  binaurally  to  the  operator’s  ears.  Actually  the 
vehicle  was  also  part  of  an  autonomous  vehicle  navigation 
project.  One  of  the  justifications  given  for  telepresence  in  such  a 
vehicle  was  the  need  for  the  operator  to  help  out  the  autonomous 
system  when  it  runs  into  trouble  and/or  asks  for  assistance.  In 
experimental  evaluations  with  the  vehicle  it  was  found  that  many 
collisions  occurred  when  a  conventional  video  display  was  used, 
whereas  the  head-controlled  stereo  display  improved  performance 
significantly. 

Synthetic  window  display 

The  head-mounted  or  helmet-mounted  display  is  not  the  only 


Figure  7.  Tachi’s  experiments  with  head-mounted  display  for 
telepresence.  Head  ,  measured  in  6  DOF  electromagnetically  or 
mechanically  (6  DOF  boom  not  shown),  drives  camera  position,  so  that 
human’s  video  display  corresponds  to  viewpoint  in  remote 
environment.  [Courtesy  S.  Tachi,  Ministry  of  Trade  and  International 
Industry,  Japan.] 


way  to  achieve  “geometrically  correct”  visual  telepresence.  The 
virtual  window  is  another  technique.  Schwartz  [39]  describes  a 
fixed  high-resolution  stereo-video  system  with  head  tracking, 
corresponding  camera  positioning,  and  image  reproduction  to 
each  eye  to  correspond  to  what  the  viewer  would  see  were  he 
looking  through  a  fixed  window.  Merritt  [40]  reports  ongoing 
research  to  utilize  both  of  these  techniques  in  a  sophisticated 
telepresence  viewing  system. 

Virtual  presence 

When  a  computer-generated  picture  is  substituted  for  the  video 
picture  and  similarly  referenced  to  the  head  orientation,  the 
viewer  can  be  made  to  feel  present  within  an  artificial  world, 
which  in  addition  to  displays  can  include  controls  (which  one 
actuates  by  moving  one’s  hand  to  the  corresponding  locus  in 
body-referenced  space).  The  term  virtual  presence  is  used  to 
describe  such  an  arrangement.  Synonyms,  some  of  which  seem 
self-contradictory,  are  virtual  environment,  artificial  reality,  and 
virtual  reality.  It  may  be  said  that  systems  to  create  virtual  reality 
are  now  a  reality.  This  can  be  attributed  to  the  availability  of 
computer  graphic  systems  which  are  able  to  generate  compelling 
object  representations  sufficiently  fast,  greatly  improved  head- 
mounted  displays  and  optics,  and  position  sensors  such  as  the 
Polhemus,  the  VPL  DataGlove,  and  the  EXOS  exoskeleton 
which  can  translate  head  and  free  limb  movements  into 
corresponding  apparent  movements  of  the  computer-generated 
objects.  Figure  8  illustrates  the  idea.  Ellis  [41]  provides  a 
sampler  of  recent  research  on  pictorial  displays  for  both  virtual 
and  “tele”  environments. 

NASA  Ames  and  USAF  Wright-Patterson  virtual 
presence  demonstrations 

Over  the  last  decade  two  virtual  environment  demonstrations 
were  mounted  simultaneously  —  one  at  NASA’s  Ames  Research 
Center  by  Fisher  and  McGreevy  and  their  colleagues  [42],  the 
other  at  the  Wright-Patterson  Air  Force  Base  Aerospace  Medical 
Research  Laboratory  by  Furness  and  his  colleagues  [43]. 

NASA’s  development  of  a  virtual  environment  workstation  was 
justified  as  an  experimental  and  developmental  tool  for  eventual 
control  of  teleoperators  with  telepresence,  as  a  control  device  for 
access  and  manipulation  of  data,  and  as  a  means  to  visualize 
physical  flow  and  other  computer-simulated  phenomena  in  three 
dimensions.  The  NASA  development  concentrated  on  achieving  a 
visually  correct  and  comfortable  head-mounted  display  and 
convincing  computer  graphics.  The  Air  Force  project  was 
explicitly  intended  to  investigate  head-mounted  djsplay  of  real¬ 
time  computer-generated  images  to  a  fighter  pilot,  who,  instead 
of  looking  out  at  the  (possibly  weather  obscured)  real 
environment,  could  look  around  to  see  a  clear  virtual  environment 
of  mountains  or  other  terrain  (labeled  as  necessary),  command 
trajectory,  threat  locations,  etc.  The  Air  Force  project  also 
concentrated  on  miniaturizing  the  head-mounted  display  system. 


HEAD  MOUNTED  DISPLAY 
(ENCLOSED) 


Figure  8.  Operator  wearing  head-mounted  visual  and  auditory  display 
and  instrumented  gloves,  experiencing  virtual  presence  in  fictional 
space  environment  generated  by  computer.  [Courtesy  NASA.] 


Virtual  acoustic  displays  may  play  an  important  role  in  virtual 
presence.  This  is  largely  due  to  the  “cocktail  party  effect”  [44], 
the  ability  to  resolve  and  identify  meaningful  sound  patterns 
spatially  even  though  their  signal  strength  is  but  a  fraction  of  the 
total  sound  energy  entering  the  ear.  On  the  basis  of  power 
spectral  transfer  functions  for  sounds  reaching  the  eardrum  from 
sources  at  different  external  locations  (distortions  due  to  head  and 
pinna  structures  as  well  as  room  configuration  and  damping 
characteristics),  an  electronic  device  called  the  Convolvatron  can 
produce  in  earphones  a  realistic  experience  of  multiple  sound 
sources  as  a  function  of  head  position  and  orientation  [45]. 

Patrick  [46]  added  a  tactile  buzzer  to  a  VPL  Data  Glove  and 
programmed  it  so  that  the  wearer  can  “reach  out  out  touch 
something”  —  in  this  case,  when  the  index  finger  or  the  thumb  or 
both  are  correctly  positioned  they  feel  a  vibrotactile  stimulus. 

Such  a  tactile  display,  however,  gives  the  impression  of  touching 
a  tuning  fork,  not  an  inert  object.  Kramer  [47]  has  done  a  similar 
experiment  with  a  mechanically  servoed  plate  which  pulls  against 
the  fingertip  as  the  fingers  close  to  create  a  virtual  touch  of  a 
virtual  object. 

Teleproprioception  by  viewing  a  computer-modeled 
virtual  environment 

One  of  the  most  promising  ways  of  achieving  teleproprioception 
is  through  computer  aiding.  In  this  case  the  computer,  when 
given  the  positions  and  orientations  of  teleoperator 
vehicle  (base),  arm  and  hand  segments,  environmental  objects, 
and  video  camera  view,  can  provide  a  synthesized  view  from  any 
position  and  orientation  selected  by  the  operator.  The  viewpoint 
is  no  longer  restricted  by  where  the  video  cameras  happen  to  be. 
The  operator  is  free  to  “roam”  arbitrarily  to  get  a  vantage  point  he 
likes,  or  to  compare  the  views  from  several  different  points, 
perhaps  using  a  joystick  or  other  controller  to  “fly”  her  viewpoint 
around  in  simulated  space. 

Das  [48],  in  the  context  of  his  above-mentioned  experiments  with 
computer-simulated  telemanipulation,  systematically  compared 
this  free  selection  technique  with  three  other  views.  A  second 
view  was  fixed  just  above  the  manipulator  base,  as  though  the 
operator  were  viewing  through  a  window  from  inside  the  vehicle. 
A  third  view  was  fixed  in  space  to  one  side  of  the  task  and  some 
distance  away.  And  a  fourth  view  was  selected  by  a  “best  view” 
algorithm.  The  algorithm  assumed  the  operator  wants  a  true 
projection  of  three  relative  distances:  from  manipulator  end  point 
to  the  target  and  to  the  two  closest  obstacles.  For  any  one  of  these 
distances,  an  equally  good  viewpoint  was  anywhere  on  the  plane 
bisecting  the  line  drawn  between  the  two  objects,  e.g.,  the  end 
point  and  the  target.  Thus  there  were  three  planes  to  consider. 


Figure  9.  Computer-controlled  “best  view’’  of  telerobot  used  by  Das. 
Lines  li,  12  ,  and  I3,  respectively,  are  the  distances  from  the 
teleoperator  to  the  two  closest  obstacles  (circles)  and  to  the  goal,  G. 
Planes  Pi,  P2  and  P3  are  perpendicular  bisectors  of  these  lines.  The 
“best”  viewpoint  is  taken  to  be  at  the  intersection  of  these  planes  (at  B 
in  the  case  shown)  because  it  provides  an  orthogonal  projection  of 
these  distances  to  the  obstacles  and  to  the  goal.  [From  Das,  ref.  48] 


The  intersection  of  the  three  planes  was  taken  to  be  the  “best” 
viewpoint.  Figure  9 

Das’  results  showed  that,  in  terms  of  speed  and  obstacle 
avoidance,  performance  was  best  when  the  operator  could  select 
the  view.  The  automatically  selected  view  and  the  fixed  point 
beside  the  task  were  less  good  for  most  subjects,  but  one  novice 
did  best  with  the  automatic  view.  The  view  “out  the  window” 
was  worst. 

One  interesting  application  of  virtual  reality  is  surgical  training, 
and  a  specific  example  of  that  is  simulation  of  arthroscopy 
(surgery  on  bones  and  joints  by  using  surgical  instruments  and 
fiber  optics  inserted  through  the  patient’s  skin).  In  such  a 
simulation,  the  arthroscopist  trainee  sees  on  a  computer  screen 
(as  though  through  an  optical  fiber  bundle  inserted  in,  say,  a  knee 
joint)  a  knife  or  tweezer  or  both  (as  though  inserted  from  another 
direction)  as  well  as  cartilage,  muscle  tissue,  and  bone.  The 
trainee  holds  a  simulated  surgical  instrument  and  makes  a  cut,  at 
the  same  time  seeing  the  knife  cut  through  cartilaginous  tissue 
and  feeling  the  viscous  reaction  forces.  When  the  “knife”  hits  the 
bone  he  sees  it  stop  and  at  the  same  time  feels  it  up  against  a  hard 
surface.  The  mechanical  design  of  such  a  multi-DOF  virtual 
environment  tool  and  the  generation  of  high-bandwidth  forces 
which  correspond  accurately  to  real  force  feedback  is  a  difficult 
problem,  as  is  the  algorithmic  modeling  of  impedance 
characteristics  for  various  kinds  of  animal  tissue.  K.  P.  Chin 
explored  this  mixture  of  visual  and  tactile  virtual  environments  in 
unpublished  experiments  performed  at  the  MIT  Human-Machine 
Systems  Laboratory. 

Design  criteria  for  visual  telepresence  and  virtual 
presence 

High-quality  visual  telepresence  or  virtual  presence  requires  that 
the  viewed  image  follow  the  head  motion  with  no  apparent  lag  or 
jitter  (this  is  a  servo-control  problem  and  has  been  hard  to  achieve 
in  existing  systems),  that  an  object  in  the  display  subtend  the 
same  retinal  angle  as  it  would  in  direct  vision,  and  that  motion 
parallax  and  other  head-motion  cues  also  correspond  to  direct 
viewing.  Other  problems  are  in  achieving  sufficient  field  of  view 
(it  should  be  at  least  60°),  depth  of  field,  correct  focal  length, 
image  separation  for  stereoscopic  fusion,  and  luminance, 
resolution,  color,  and  other  image-quality  factors,  particularly  at 
the  fovea.  When  the  image  is  computer-generated,  additional 
problems  lie  in  achieving  sufficient  image-generation  speed  and 
frame  rate,  grayscale,  and  variable  accommodation  (in  contrast  to 
fixed  focus  at  infinity).  As  one  might  expect,  there  are  also 
serious  problems  of  cost,  size,  and  weight. 

It  is  natural  to  seek  an  objective  measure  or  criterion  that  can  be 
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used  to  say  that  telepresence  or  virtual  presence  have  been 
achieved.  However,  telepresence  (or  virtual  presence)  is  a 
subjective  sensation,  much  like  mental  workload,  and  it  is  a 
mental  model  —  it  is  not  so  amenable  to  objective  physiological 
definition  and  measurement.  Some  might  assert  that  a  subjective 
report  from  the  person  having  the  experience  is  the  only  measure. 
An  objective  criterion  might  be  a  test  analogous  to  that  for 
computer  intelligence  attributed  to  Alan  Turing:  if  the  observer 
cannot  reliably  tell  the  difference  between  telepresence  (or  virtual 
presence)  and  direct  presence,  then  the  telepresence  (virtual 
presence)  has  been  fully  achieved.  A  practical  criterion  of 
telepresence  proposed  by  Held  and  Durlach  [49]  is  the  degree  to 
which  the  observer  responds  in  a  natural  way  to  unexpected 
stimuli  —  e.g.,  by  blinking  her  eyes  or  ducking  her  head  when 
he  sees  that  an  object  is  about  to  hit  her.  We  are  far  from  meeting 
this  strict  criterion  in  most  applications. 

Three  independent  determinants  of  the  sense  of 
presence 

In  consideration  of  what  Held  and  others  have  suggested, 
Sheridan  [50]  proposed  that  there  are  three  principal  and 
independent  determinants  of  the  sense  of  presence:  extent  of 
sensory  information  (the  transmitted  bits  of  information 
concerning  a  salient  variable  to  appropriate  sensors  of  the 
observer),  control  of  relation  of  sensors  to  environment  (e.g., 
ability  of  the  observer  to  modify  her  viewpoint  for  visual  parallax 
or  visual  field,  or  to  reposition  her  head  to  modify  binaural 
hearing,  or  ability  to  perform  haptic  search),  and  ability  to  modify 
physical  environment  (e.g.,  the  extent  of  motor  control  to  actually 
change  objects  in  the  environment  or  their  relation  to  one 
another). 

These  determinants  may  be  represented  as  three  orthogonal  axes 
(see  Figure  10),  since  the  three  can  be  varied  independently  in  an 
experiment.  Perceived  extent  of  sensory  information  is 
sometimes  regarded  as  the  only  salient  factor.  Lines  of  constant 
information  communicated  are  suggested  in  the  figure  to  indicate 
that  the  “extent  of  sensory  information”  is  a  much  greater 
consumer  of  information  (bits)  than  the  two  control  components, 
“control  of  sensors”  and  “ability  to  modify  environment.” 

Given  the  three  independent  determinants  of  presence,  the  larger 
research  challenge  is  the  determination  of  the  dependent  variables: 
sense  of  presence  (as  measured  by  subjective  rating  and  by  the 
objective  measures  suggested  above),  objective  training 
efficiency,  and  ultimate  task  performance. 


sensors 

Figure  1 0.  Three  components  of  presence.  Hypothetical  lines  of 
constant  information  suggest  that  for  purposes  of  providing  a  sense 
of  presence,  information  channels  are  better  used  for  control  of 
sensors  and  modification  of  the  environment  than  for  higher 
resolution  displays.  [From  Sheridan,  ref.  50] 


Jex’s  criteria  for  “feel”  of  hand  controls  and  time 
delay  in  simulators 

The  above  discussion  of  telepresence  and  virtual  presence  was 
based  primarily  on  visual  considerations.  What  can  be  said  of 


proper  telepresence  conditions  for  force  feedback?  There  is 
considerable  experience  with  force  feedback  in  aircraft  and 
automobile  simulators,  where  reproducing  the  feel  of  hand 
controls’  feedback  is  a  high  priority. 

Jex  [51],  based  on  much  experience  with  aircraft  and  automobile 
simulators,  posited  the  following  four  critical  tests  for  achieving 
virtual  reality  in  the  “feel”  of  hand  controls  (control  sticks  in 
aircraft,  steering  wheels  in  automobiles  —  what  Jex  calls 
“manipulanda”): 

•  With  all  other  simulated  forces  set  to  zero,  when  the  mass  or 
inertia  of  the  simulated  hand  control  is  set  to  zero,  it  should  feel 
like  a  stick  of  balsa  wood  (i.e.,  have  negligible  lag,  friction, 
jitter,  or  forces)  up  to  the  highest  frequency  that  a  finger  grip  can 
impose,  or  about  7  Hz. 

•  When  pushed  against  simulated  hard  stops,  the  hand  control 
should  stop  abruptly,  with  no  sponginess,  and  it  should  not  creep 
as  force  continues  to  be  applied. 

•  When  set  for  pure  Coulomb  friction  (i.e.,  within  a  non¬ 
centering  hysteresis  loop),  the  hand  control  should  remain  in 
place,  without  creep,  sponginess  or  jitter,  even  when  repeatedly 
tapped. 

•  When  set  to  simulate  a  mechanical  centering  “detent”  and  moved 
rapidly  across  the  detent,  the  force  reversal  should  be  crisp  and 
give  a  realistic  “clunk”  with  no  perceptible  lag  or  sponginess. 

Jex  has  also  concluded  that  for  a  wide  range  of  simulations  in 
which  operator  steering  of  a  vehicle  is  involved,  in  order  to  keep 
mental  workload  and  performance  within  acceptable  bounds,  any 
simulation  delay  artifact  must  be  less  than  about  one-fourth  of  the 
effective  operator  response  delay. 


MANUAL  CONTROL 

Classical  models  of  manual  control  and  movements 

Since  roughly  1950  there  has  been  much  effort  devoted  to  using 
conventional  linear  control  theory  to  model  simple  manual  control 
systems  (Figure  1 1),  where  the  human  operator  is  the  sole  in-the- 
loop  control  element  and  the  controlled  process  can  be 
represented  by  linear  differential  equations.  A  primary  motivation 
for  this  work  was  the  need  to  establish  predictive  models  for 
control  of  aircraft,  where  having  good  differential  equation 
models  for  the  controlled  process  (airframe  plus  control 
electromechanics  in  this  case)  was  of  no  use  unless  models  of  the 
pilot  were  factored  in  as  well.  Initially  it  was  believed  that  an 
independent  model  of  the  pilot  was  appropriate,  so  that  the  pilot 
model  could  then  be  combined  with  whatever  controlled  process 
was  of  interest.  This  was  soon  found  to  be  impractical,  since  the 
characteristics  of  the  human  operator  proved  to  be  very  much 
dependent  upon  the  controlled  process,  varying  to  compensate 
for  the  controlled  process  so  as  to  stabilize  the  closed-loop 
system  and  provide  satisfactory  transient  response. 


tradeoffs  between 
error,  time  and 


Figure  11.  Simple  compensatory  manual  control  paradigm.  Action  u 
causes  simple  nulling  of  error  e  between  current  measured  state  x 
and  desired  state  (goal). 


It  was  then  proposed  to  model  the  human  operator  and  the 
controlled  process  as  a  single  forward  loop  element,  to  combine 
the  two  upper  blocks  of  figure  11.  The  idea  was  that  there  would 
be  only  minor  variation  of  the  combined  human  and  process  from 
application  to  application.  This  approach  proved  very  successful. 
The  result  is  the  simple  crossover  model  of  McRuer  and  Jex  [26] 
which  has  the  form 
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x/e  =  Ke-j®T  /  jco. 

This  is  essentially  a  combined  pure  time  delay  and  integrator,  to 
being  frequency.  The  (small)  variations  of  parameters  K  and  T 
are  well  established  in  the  literature.  What  is  important  here  is  the 
idea  of  a  model  in  which  human  operator  plus  controlled  process 
is  what  is  invariant,  not  the  human  operator  per  se. 

Hick  [52]  proposed  a  model,  based  on  Shannon’s  [53] 
information  theory,  for  the  time  it  takes  to  choose  which  of 
several  alternative  movements,  i,  to  make  when  the  choice  is 
based  on  an  immediately  displayed  signal  calling  for  that  move, 
and  the  move  time  itself  is  brief  and  constant: 

Hchoice  =  T]  Pi  log2(l/pi),  Tchoice  =  Ct  +  P  Hchoice 

where  Hchoice  is  information  in  bits,  pi  is  the  probability  of 
signal  i,  Tchoice  is  the  time  required  to  choose,  and  a  and  P  are 

scaling  constants  dependent  on  task  conditions,  a  includes  at 
minimum  the  base  reaction  time  for  making  the  slightest  hand 
movement  in  response  to  a  visual  stimulus. 

Fitts  [54]  also  used  the  information  measure  for  his  model 
of  the  time  required  for  making  a  discrete  arm  movement : 

Hmove  =  log2  (2A  /  B),  Tm0ve  -  cn  +  P  Hmove 

where  (see  Figure  12)  Hm0ve  is  information  in  bits  (sometimes 
also  called  index  of  difficulty),  A  is  the  distance  moved,  B  is  the 
tolerance  to  within  which  the  move  must  be  made  (for  Fitts’ 
experiment  a  tap  between  two  lines),  T  is  task  completion  time, 
and  a  and  p  are  again  scaling  constants,  different  for  different 
conditions.  Fitts  probably  did  not  realize  what  wide  application 
the  model  would  find.  When  applied  to  simple  one-dimensional 
movements  to  within  tolerances,  the  model  has  withstood  the  test 
of  time  and  been  robust  over  a  wide  range  of  A’s,  B’s,  and  other 
task  conditions  such  as  bare-handed  vs.  master-slave  manipulator 
It  was  successfully  fitted  to  experimental  data  in  a  number  of  the 
studies  described  above.  However,  like  so  many  elegant  models 
for  human  behavior,  Fitts’  model  breaks  down  for  more  complex 
manipulations. 


Figure  12.  Fitts’  experiment  for  moving  a  given  distance  to  within  a 
given  tolerance.  [From  Sheridan,  ref.  1] 


If  a  person  must  first  make  a  choice  and  then  move,  a 
first-order  model  for  time  required  will  then  be 

T  total  =  Tchoice  +  Tmove. 

Thompson  [55],  in  his  studies  of  manipulation,  showed  that  the 
time  required  to  mate  one  part  to  another  was  a  function  of  the 
degrees  of  constraint  (the  number  of  positions  and  orientations 
that  simultaneously  have  to  correspond  before  the  final  mating 
could  take  place).  Figure  13  illustrates  the  idea  of  degrees  of 
constraint.  Data  based  on  both  the  Fitts  and  the  Thompson 
measures  will  be  shown  later  when  time-delayed  manipulation  is 
discussed. 

Operator  hand  controls 

The  most  popular  operator  hand  controls  are  the  articulated 
master  arm  and  the  joystick. 
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Figure  13.  Thompson's  degrees  of  constraint.  [From  Thompson,  ref. 
55] 


Most  articulated  master  arms  have  been  used  for  master-slave 
positioning.  Usually  these  have  been  kinematically  isomorphic  to 
the  slave  in  construction.  They  need  not  be,  however,  so  long  as 
they  have  the  same  number  of  DOF  as  the  slave  and  as  long  as 
the  end-point  direction  of  motion  of  the  master  and  the  slave 
correspond  —  the  computer  can  do  the  coordinate 
transformations.  The  master  should  be  sized  to  the  convenience 
of  the  operator  —  it  need  not  be  the  same  size  as  the  slave.  Its  full 
range  of  motion  must  conform  both  to  the  available  space  and  to 
the  operator’s  reach;  at  the  same  time  there  must  be  enough 
movement  that  small  random  muscle  movements  do  not  produce 
significant  command  signals.  These  constraints  on  both  ends 
limit  the  dynamic  range.  Master  arms  as  small  as  1  foot  in  total 
length  are  in  use.  Master-slave  position  control  provides  quick 
and  natural  repositioning,  but  relatively  poor  ability  to  move  very 
slowly. 

The  joystick  is  commonly  available  in  up  to  three  degrees  of 
freedom,  is  normally  spring  centered,  and  commands  an  end- 
effector  (or  vehicle)  rate  proportional  to  joystick  displacement.  It 
can  also  command  acceleration.  There  is  usually  kinematic 
resolution  by  computer,  so  that  end-effector  movement  direction 
corresponds  to  joystick  movement  independent  of  the  arm 
configuration.  A  slight  electrical  dead-zone  is  usually  added  to 
prevent  inadvertent  drifting  of  the  arm  in  any  particular  direction. 
The  joystick  permits  high  positioning  accuracy,  but  at  the  cost  of 
relatively  slow  repositioning  movements.  In  six-  DOF 
applications  two  joysticks  have  typically  been  used:  one  to 
command  translation  in  three  DOF  and  one  to  command  rotation 
in  three  DOF.  There  have  been  several  six-axis  joystick  designs, 
one  of  which  (developed  by  the  German  firm  DLR)  uses  optical 
transducers,  and  a  similar  device  called  the  “Space  Ball”  in  the 
US.  Mostly  these  are  isometric  devices  (very  stiff  “force-sticks”). 
There  has  been  some  question  about  how  well  an  operator  can 
use  such  a  device  to  control  all  six  axes  at  once. 
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Massimino  et  al.  [56]  experimentally  measured  human  tracking 
capability  using  such  a  six-axis  “sensor-ball”  to  keep  two  virtual 
spheres  aligned  horizontally  and  vertically  and  of  the  same  size 
(front-back  indication)  and  to  keep  tcross  hairs  on  the  cursor  and 
the  target  together.  They  found  that  translational  movement 
perpendicular  to  the  display  screen  produced  significantly  more 
RMS  error  than  did  horizontal  or  vertical  translations. 
Performance  in  rotational  DOF  did  not  differ  significantly  from 
one  to  another.  Rate  control  was  better  than  acceleration  control, 
as  expected.  With  more  simultaneous  DOF,  performance  on  any 
one  axis  deteriorated.  Figure  14  illustrates  the  translation  results. 
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Figure  14.  Massimino’s  results  of  tracking  with  six-axis  sensor  ball. 
Subjects  simultaneously  tracked  in  all  6  degrees  of  freedom.  The 
graph  shows  the  component  translation  errors  in  x,  y  and  z.  Data 
points  are  averages  of  six  subjects.  Vertical  lines  indicate  one 
standard  deviation.  [From  Massimino  et  al.,  ref.  56] 


Some  newly  developed  commercial  sensors  that  can  be  used  to 
do  six-axis  positioning  have  the  advantage  over  the  conventional 
master-arm  that  they  can  be  positioned  freely  in  space  without  a 
mechanical  link  to  a  fixed  base.  The  Polhemus  sensor  is  a  six- 
DOF  electromagnetic  system  consisting  of  a  transmitter  and  a 
receiver,  each  slightly  bigger  than  a  cubic  centimeter.  When  one 
is  moved  relative  to  the  other,  provided  they  are  not  separated  by 
more  than  a  meter  and  provided  there  is  little  interference  from 
nearby  metal  objects,  the  change  in  their  relative  position  is 
measured  in  all  six  DOF  to  within  several  millimeters  in 
translation  and  roughly  1°  in  rotation. 

The  VPL  DataGlove  is  a  nylon  glove  worn  by  the  operator.  It  has 
optical  fibers  embedded  on  the  dorsal  side  of  each  finger  and  the 
thumb.  When  any  joint  flexes,  light  passing  through  the  optical 
fiber  serving  that  joint  is  reduced  because  of  scattering,  and  this 
is  a  measure  of  the  flexure  of  the  optical  fiber.  Unfortunately  the 
glove  tends  to  slip  on  the  skin,  the  bends  of  the  fibers  do  not 
coincide  one-to-one  with  the  operator’s  finger-joint  rotations,  and 
both  may  have  different  kinematics  from  the  teleoperator  hand 
being  used.  Therefore  the  control  of  teleoperation  is  neither 
precise  nor  consistent,  and  it  may  be  necessary  to  perform  rather 
complex  calibrations  in  order  for  satisfactory  open-loop  mapping 
from  operator  hand  pose  to  teleoperator  hand  pose  to  be  achieved 
[57][58].  The  Exos  articulated  multi-DOF  hand  goniometer 
(Figure  15)  serves  a  similar  function  but  consists  of  a  pin-jointed 
multi-segment  exoskeleton  worn  on  the  hand,  with  Hall-effect 
sensors  to  measure  joint  rotations.  It  is  bulkier  but  more  accurate 
than  the  DataGlove. 

Using  other  body  limbs  and  the  head  to  signal  the 
computer 

Body  limbs  other  than  hands  have  been  used  as  means  of  control 
,  the  most  common  example  being  the  clutch,  brake,  and 
accelerator  pedals  of  automobiles.  The  lower  limbs,  because  of 
their  mass,  are  not  the  fastest  means  of  control,  but  they  do  give 
the  advantage  of  large  force  capability,  which  was  a  good  reason 
for  assigning  them  to  operate  brake  and  clutch  pedals  prior  to  the 
time  of  power  assist. 


Figure  15.  EXOS  hand  goniometer.  [Courtesy  of  Exos  Corp.j 


Head  motion  and  eye  motion  are  other  means  for  the  operator  to 
signal  the  computer.  Head  motion  can  be  measured  by 
mechanical  linkages  attached  to  helmets  or  by  electromagnetic  or 
optical  trackers.  Eye  tracking  instruments  are  now  readily 
available  and  of  high  quality,  and  are  widely  used  to  measure  eye 
scan  paths.  Mostly  these  follow  a  pattern  of  50-msec  saccades 
(rapid  movements  which  are  quite  unpredictable  in  direction  and 
extent  of  movement  and  which  seem  to  depend  on  the 
information  “of  interest”  in  the  image)  followed  by  200-500- 
msec  dwell  periods,  during  which  the  brain  presumably  reads  in 
the  information. 

Operator-resolved  and  task-resolved  coordinates 

Operator-resolved  manipulation  means  that  the  operator  can  move 
or  rotate  her  own  hand  and  have  the  teleoperator  end  effector 
move  or  rotate  in  the  same  direction,  independent  of  the  kinematic 
configuration  of  the  master  or  slave  arm  segments  or  joint 
positions.  This  is  achieved  by  using  the  Jacobian  [59]  for 
resolving  the  velocity  of  the  slave  in  correspondence  to  the 
velocity  of  the  master. 

Task-resolved  manipulation  means  that  the  operator  can  order  the 
teleoperator  end  effector  to  move  or  apply  force  in  a  coordinate 
system  referenced  to  a  normal  to  the  surface  of  a  large  object  or 
environmental  structure  such  as  a  ship  or  a  pipe  [60].  This 
requires  sensing  that  surface  in  the  process  of  manipulating  and 
continually  performing  coordinate  transformations  to  update  the 
axes  with  respect  to  which  the  operations  are  being  done  This  is 
an  extension  of  end-point  resolution  - —  ability  to  command  the 
finger  to  move  in  a  desired  trajectory  without  having  to  worry 
about  how  to  move  all  the  joints  between  the  finger  and  the 
shoulder. 


COPING  WITH  TIME  DELAY 
Why  there  is  a  problem 

Continuous  teleoperation  in  earth  orbit  or  deep  space  by  human 
operators  on  the  earth’s  surface  is  seriously  impeded  by  signal 
transmission  delays  imposed  by  limits  on  the  speed  of  light  (radio 
transmission)  and  computer  processing  at  sending  and  receiving 
stations  and  satellite  relay  stations.  For  vehicles  in  low  earth 
orbit,  round-trip  delays  (the  time  from  sending  a  discrete  signal 
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until  any  receipt  of  any  feedback  pertaining  to  the  signal)  are 
minimally  0.4  seconds;  for  vehicles  on  or  near  the  moon  these 
delays  are  typically  3  seconds.  Usually  the  loop  delays  are  much 
greater,  approaching  6  seconds  in  the  case  of  the  earth-orbiting 
space  shuttle  because  of  multiple  up-down  links  (earth  to  satellite 
or  the  reverse)  and  the  signal  buffering  delays  which  occur  at 
each  device  interface.  A  similar  problem  is  encountered  with 
remote  control  in  the  deep  ocean  from  the  surface  if  acoustic 
telemetry  is  employed  to  avoid  dragging  miles  of  heavy  cable. 
Because  sound  transmission  is  limited  to  around  1700  m/sec  in 
water,  communicating  over  a  1700-rn  distance  poses  a  2-sec 
round-trip  delay. 

Continuous  closed-loop  control  over  a  finite  time  delay  is  not 
possible,  because  any  energy  entering  the  loop  at  such  a 
frequency  that  half  a  cycle  is  equal  to  the  time  delay  will  result  in 
positive  feedback  rather  than  negative,  so  that  if  the  loop  gain 
exceeds  unity  at  this  frequency  (which  it  normally  would  at  low 
frequency)  there  is  an  inherent  instability.  Of  course  in  the  case  of 
supervisory  control,  wherein  commands  are  sent  by  the  human 
operator  through  the  time  delay  to  a  computer,  the  computer  then 
implements  the  commands  by  closing  loops  local  to  itself, 
reporting  back  to  the  supervisor  when  the  task  is  completed.  The 
computer’s  local  loop  closure  has  no  delay  in  it  and  therefore 
causes  no  instability.  Nor,  because  of  the  intermittent  nature  of 
the  supervisor’s  control,  does  the  delay  in  her  command- 
feedback  loop  cause  instability. 

Early  experiments  with  time  delay 

With  such  delays  in  a  continuous  telemanipulation  loop,  it  has 
been  shown  experimentally  that  the  time  for  a  human  operator  to 
accomplish  even  simple  manipulation  tasks  can  increase 
manyfold,  depending  upon  the  time  delay  and  the  complexity  of 
the  task.  This  is  because  the  human  operator,  in  order  to  avoid 
instability  (which  is  quite  predictable  from  simple  control  theory), 
must  adapt  what  has  come  to  be  called  a  “move  and  wait 
strategy,”  wherein  he  commits  to  a  small  incremental  motion  of 
the  remote  hand  or  vehicle,  stops  while  waiting  (the  round-trip 
delay  time)  for  feedback,  then  commits  to  another  small  motion, 
and  so  on. 

Ferrell  [61]  was  the  first  to  demonstrate  experimentally  the 
predictability  of  teleoperation  task  performance  as  a  function  of 
the  delay,  the  ratio  of  movement  distance  to  required  accuracy  , 
and  other  aspects  of  delayed  feedback  in  teleoperation.  Ferrell’s 
results  (Figure  16)  are  for  simple  two-axis-plus-grasp 
manipulations  on  a  table.  Black  [62]  performed  similar 
experiments  with  a  conventional  six-axis-plus-grasp  master-slave 
manipulator. 

Thompson  [55]  showed  how  task-completion  time  was  affected 
not  only  by  time  delay  but  also  by  degrees  of  constraint  (see 
Figure  13).  Thompson’s  experimental  results  are  shown  in 
Figure  17. 

This  problem  has  discouraged  control  of  space  vehicles  from  the 
ground.  However,  as  more  and  more  devices  are  put  in  space, 
the  requirements  increase  for  humans  to  perform  remote 
manipulation  and  control,  and  if  this  can  be  done  entirely  from 
earth  there  are  great  savings  in  dollars  and  risk  to  life. 

Early  predictor  displays 


Index  of  difficulty  I  in  bits 


Figure  16.  Ferrell’s  results  for  time-delay  in  telemanipulation. 
Experiments  were  performed  in  simple  two-DOF  grasp-and-place 
tasks  with  various  accuracy  requirements  (Fitts'  index  of  difficulty)  and 
pure  time  delays.  [From  Ferrell,  ref.  61] 


“Predictor  displays,”  where  cursors  or  other  indications  driven 
by  a  computer  are  extrapolated  forward  in  time,  are  of  two  types. 
A  first  is  based  upon  current  state  and  time  derivatives  —  i.e., 
Taylor-series  extrapolation.  A  second  involves  inputting  current 
state  and  time  derivatives,  as  well  as  expected  near-future  control 
signals,  into  a  model  [63].  Such  displays  have  been  employed  in 
gunsights,  on  ships  and  submarines,  and  as  “head-up”  optical 
landing  aids  for  aircraft  pilots.  When  there  is  significant 
transmission  delay  (say  more  than  0.5  sec)  and  a  slow  frame  rate 
(say  less  than  one  frame  per  4  seconds),  a  predictor  display  can 
be  very  useful.  Both  of  the  latter  conditions  are  likely  to  be 
present  with  long-distance  acoustic  communication. 

Verplank  [64]  implemented  an  experimental  predictor  of  the 
second  type  for  a  simulated  planetary  rover.  A  computer  model  of 


Figure  17.  Thompson’s  results  for  time  delay  and  degrees  of 
constraint  (defined  in  figure  13).  Averaged  times  include  transport  of 
peg  to  hole,  positioning,  and  inserting.  [From  Thompson,  ref.  55] 


the  vehicle  was  repetitively  set  to  the  present  state  of  the  actual 
system,  including  the  present  control  input,  then  allowed  to  run  at 
roughly  100  times  real  time  for  a  few  seconds  before  it  was 
updated  with  new  initial  conditions.  During  each  fast-time  run,  its 
response  was  traced  out  in  a  display  as  a  prediction  of  what 
would  happen  over  the  next  time  interval  (say  several  minutes)  “if 
I  keep  doing  what  I’m  doing  now.”  Such  techniques  are  adequate 
for  continuous  control  of  single-entity  or  “rigid  body”  vehicles, 
but  not  for  telemanipulation,  where  it  is  necessary  to  predict. 


COMPUTER  PREDICTION 
(immediate  response) 


feedback  communication 


Distance  :  20  (in,) 


Figure  18.  Noyes’ telemanipulation  predictor  display,  (a)  Diagram  of 
experimental  setup,  (b)  Photograph  of  stick  figure  arm  superposed 
on  video  screen.  [From  Noyes  ,  ref.  65] 


relative  to  the  environment,  the  simultaneous  positions  of  a 
number  of  parts  —  i.e.,  a  spatial  configuration  in  multiple 
degrees  of  freedom,  not  just  a  single  point. 


Figure  19.  Hashimoto  results  for  predictor  display  evaluation  in  simple 
task  of  repositioning  a  block  20  inches  to  within  a  one  inch  tolerance. 
Data  shown  are  for  one  subject.  Brackets  are  standard  error  of  the 
mean  for  repeated  trials.  [From  Hashimoto  et  al. ,  ref.  68] 


A  prediction  architecture  proposed  by  Hirzinger  et  al.  [69] 
includes  this  notion  (Figure  20)  as  well  as  dynamic  prediction. 
The  stick-figure  overlay  on  the  delayed  video  is  driven  by  a 
dynamic  model  (whereas  Noyes  et  al.  used  a  kinematic  model). 
In  the  figure  this  is  constituted  by  the  sum  of  the  A  and/or  B 
feedback  coefficients  operating  on  correspondingly  delayed 
commands.  In  the  middle  of  the  diagram  is  the  implementation  of 
the  canonical  first-order  x(k+l)  =  Ax{k)  +  Bu(k),  where  k 
corresponds  to  what  is  going  on  instantaneously  with  the  space 
telerobot.  The  x(/r+l)  estimate  is  corrected  in  the  usual  way  by 
Kalman  gain-multiplied  discrepancy  between  estimated  y{k-nd) 
and  the  corresponding  actual  downlink  signal.  The  delay  line  on 
the  right  side  is  required  to  estimate  y(k-nd).  By  estimating  x(k) 
—  i.e.,  what  is  happening  in  space  —  activities  such  as 
rendezvous  and  docking  can  be  coordinated  with  clock- 
determined  events  which  are  not  under  the  control  of  this  human 
operator. 


Noyes  [65]  built  the  first  predictor  display  for  telemanipulation, 
using  newly  commercially  available  computer  technology  for 
superposing  artificially  generated  graphics  on  to  a  regular  video 
picture.The  video  picture  was  a  (necessarily  simulated)  time- 
delayed  picture  from  the  remote  location,  generated  as  a  coherent 
frame  (snapshot)  so  that  all  picture  elements  in  a  single  scan  were 
equally  delayed.  (Otherwise  the  part  of  the  screen  refreshed  last 
would  be  delayed  more  than  the  part  refreshed  first.)  As  shown 
in  Figure  18,  the  predictor  display  was  a  line  drawing  of  the 
“present”  configuration  of  the  manipulator  arm  or  vehicle  or  other 
device.  The  latter  was  generated  by  using  the  same  control 
signals  that  were  sent  to  the  remote  manipulator  (device)  to  drive  ( 
a  kinematic  model  of  it.  The  computer  model  was  drawn  on  the  = 
video  display  in  exactly  the  same  location  where  it  would  actually 
be  after  a  one-way  time  delay  and  where  it  would  be  seen  to  be 
on  the  video  after  one  round-trip  time  delay.  Since  the  graphics 
were  generated  in  perspective  and  scaled  relative  to  the  video 
picture,  if  one  waited  at  least  one  round-trip  delay  without 
moving,  both  the  graphics  model  and  video  picture  of  the 
manipulator  (device)  could  be  seen  to  coincide.  The  effectiveness 
of  these  techniques  was  demonstrated  for  simple  models  of  the 
manipulator  arm  and  simple  tasks  [65][66][67j[68].  With  such  a 
display,  operators  could  “lead”  the  actual  feedback  and  take  larger 
steps  with  confidence,  reducing  task  performance  time  by  50% 
(Figure  19). 

Two,  more  elaborate,  predictor  instruments 

When  the  motion  of  vehicles  or  other  objects  not  under  the 
operator’s  control  can  be  predicted,  e.g.,  by  the  operator’s 
indicating  on  each  of  several  successive  frames  where  certain 
reference  points  are,  these  objects  can  be  added  to  the  predictor 
display.  With  any  of  these  planning  and  prediction  aids,  the 
display  can  be  presented  from  any  point  of  view  relative  to  the 
manipulator  or  vehicle  —  which  is  not  possible  with  the  actual 
video  camera. 


Figure  20.  Hirzinger’s  predictor  incorporating  adaptive  model.  [After 
Hirzinger  et  al.,  ref.  69] 


Another  predictor  instrument  was  developed  by  Cheng  [70]  as  an 
aid  to  human  operator  control  of  the  Woods  Hole  Oceanographic 
Institution’s  remotely  operated  submersible  Argo.  Essentially  the 
latter  is  a  heavy  vehicle  suspended  and  passively  towed  by  a  very 
long  cable  (up  to  6000  m)  from  a  support  ship.  The  time  constant 
for  changes  in  control  from  the  ship  to  become  manifest  in  the 
position  of  the  submersible  is  of  the  order  of  10  minutes.  To 
predict  the  submersible’s  trajectory  in  latitude  and  longitude  from 
steering  control  actions  performed  on  the  ship,  the  model  for  the 
predictor  must  include  the  submersible,  the  cable,  and  the  ship 
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GRAPHICAL  DISPLAY,  showing  predicted 
successive  positions  of  ship  (a.b.c)  and 
submersible  (A.B.C)  at  successive  equal 
time  intervals  after  speedup  and  turn. 


Figure  21 .  Cheng’s  adaptive  predictor  for  towed  submersible.  [From 
Cheng,  ref.  70] 


(all  fairly  nonlinear),  and  must  account  for  both  wind  and  water 
current  disturbances.  The  cable  was  the  most  difficult  to  model, 
but  it  was  found  that  a  relatively  simple  linear  model  whose 
parameters  are  continuously  updated  (see  Figure  21)  does  a  rathei 
good  job.  In  simulation  trials,  such  a  model  cut  the  error  in 
following  a  given  trajectory  to  one-third.  With  the  predictor 
display,  human  operator  control  actions  were  at  significantly 
lower  levels  of  thrust  than  without  the  predictor,  a  result 
consistent  with  theoretical  analysis  which  suggested  that  the 
predictor  effectively  lowers  the  gain  and  increases  damping. 

Time-delayed  force  feedback 


Figure  22.  Buzan’s  tasks  for  time-delayed  force  feedback 
experiments.  In  task  A,  human  subject  was  to  reach  out  and  grasp 
block  (which  was  freely  floating  in  space)  without  inadvertently 
accelerating  it  out  of  reach  before  grasp  could  be  achieved.  In  task  B, 
human  was  to  push  block  to  center  of  spring-clamp  until  light  static 
friction  held  it  there  and  not  let  it  pop  out  the  other  side.  Models  used 
to  generate  predictor  displays  were  simplifications  of  (simulated)  real¬ 
time  tasks  (e.g.,  no  static  friction  in  task  B).  [From  Buzan,  ref.  72] 


Buzan  [72] [73]  evaluated  the  latter  approach  experimentally.  He 
employed  an  open-loop  model-based  prediction  to  drive  both  a 
visual  predicted-position  display  and  a  force  exerted  back  on  the 
operator  through  a  master  positioning  arm.  He  did  his 
experiments  with  a  one-DOF  teleoperator  system,  a  3-second 
time  delay,  and  two  challenging  computer-simulated  tasks.  The 
first  task  was  to  extend  the  arm  to  make  contact  with  (and 
unavoidably  accelerate)  a  floating  mass,  then  grasp  it  with  a 
discrete  action  (an  additional  “half'  DOF)  before  it  “got  away.” 
The  second  task  was  to  push  an  object  into  a  “stiff  slot”  with 
enough  force  to  get  it  in  and  have  static  friction  hold  it  there,  but 
not  so  much  force  that  it  goes  right  out  the  other  side.  Figure  22 
illustrates  the  second  task. 

Buzan  tried  three  force-feedback-display  techniques.  In  one, 
which  he  called  direct  force  feedback,  he  simply  presented  the 
predicted  force  (but  not  the  delayed  “real”  force)  to  the  active 
hand,  the  hand  commanding  the  teleoperator  position.  In  a 
second  method,  which  he  called  dual  force  feedback,  he 
presented  the  delayed  force  to  an  inactive  hand  and  the  predicted 
force  to  the  active  hand.  In  the  third  display  technique,  which  he 
called  complimentary  force  feedback,  he  presented  to  the  active 
hand  the  sum  of  a  low-pass-filtered  delayed  force  feedback  and  a 
high-pass-filtered  predicted  force  feedback. 


Force  feedback  with  time  delay  is  a  different  problem  from  that 
with  visual  feedback.  Ferrell  [71]  showed  that  it  is  unacceptable 
to  feed  resolved  force  continuously  back  to  the  same  hand  that  is 
operating  the  control.  This  is  because  the  delayed  feedback 
imposes  an  unexpected  disturbance  on  the  hand  which  the 
operator  cannot  ignore  and  which,  in  turn,  forces  an  instability  or 
the  process.  With  visual  delay  the  operator  can  ignore  the 
disturbance  and  can  avoid  instability  by  a  move-and-wait  strategy 
or  by  supervisory  control  [61]. 


Buzan’s  results  showed,  among  other  things,  that  end-point 
impedance  made  a  big  difference  in  these  tasks.  The  contact-and- 
grasp  task  was  easiest  with  a  soft  end  point  compliance,  while  the 
slot  task  favored  a  stiff  end-point.  Buzan  also  found  that  the 
complementary  force  feedback  proved  difficult  to  use.  When  the 
visual  predictor  was  used  and  was  perfect,  the  predicted  force 
feedback  had  a  negligible  effect  on  performance.  When 
telemanipulation  was  blind,  both  the  direct  and  the  dual  force 
feedback  worked  quite  well,  enabling  the  operator  to  do  the  tasks 
where  he  otherwise  could  not. 


Since  Ferrell’s  experiments  there  have  been  various  proposals, 
the  simplest  of  which  is  to  display  force  feedback  in  visual  form 
on  a  computer  display.  Alternatively  the  force  feedback  can  be  to 
the  hand  that  is  not  on  the  master  hand  or  joystick.  Another 
suggestion  has  been  to  feed  back  disturbances  greater  than  a 
certain  magnitude  to  the  controlling  hand  for  a  brief  period,  at  the 
same  time  cutting  off  or  reducing  the  loop  gain  to  below  unity, 
and  subsequently  to  reposition  the  master  to  where  it  was  at  the 
start  of  the  event.  Finally,  there  is  the  possibility  of  predicting  the 
force  feedback  to  compensate  for  the  delay,  and  feeding  the 
predicted  force  but  not  the  real-time  force  back  to  the  operator’s 
hand. 


Sensory  substitution 

As  is  well  known,  when  there  is  force  feedback  in  a  control  loop 
with  time  delay,  the  force  reflected  to  the  hand,  unless  the  hand  is 
infinitely  stiff,  simply  reenters  the  control  loop,  producing  an 
instability,  provided  the  gain  is  greater  than  unity.  A 
circumvention  of  this  is  to  have  the  feedback  of  force  be  other 
than  an  actual  unbalanced  physical  force  to  the  hand.  Massimino 
[74]  tested  this  concept  for  the  teleoperation  task  of  inserting  a 
four-sided  peg  into  a  four-sided  hole.  Using  in  one  case 
vibrators  on  the  skin  (four  vibrators  in  a  square,  the  amplitue  of 
vibration  corresponding  to  the  interactive  force  between  the  peg 
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and  the  corresponding  side  of  the  hole),  and  in  another  case 
auditory  tones  (high  and  low  pitch  for  contact  with  top  ond 
bottom  of  hole,  left  and  right  ears  for  contact  with  left  and  right 
sides  of  the  hole).  Results  showed  clear  advantages  of  sensory 
substitution  in  either  auditory  or  vibratory  form  when  time  delay 
otherwise  would  cause  instability. 

Time  and  space  desynchronization  in  a 
predietor/planning  display 

Conway,  Volz,  and  Walker  [75]  extended  the  predictor  idea  of 
Noyes  and  Sheridan  [66]  and  combined  it  with  a  planning  model 
in  what  they  call  “disengaging  time-control  synchrony  using  a 
time  clutch”  and  “disengaging  space  control  synchrony  using  a 
position  clutch.”  In  their  scheme,  the  time  clutch  allows  the 
operator  to  disengage  synchrony  with  real  time,  to  speed  up 
making  inputs  and  getting  back  simulator  responses  for  easy 
maneuvers  and  to  slow  down  the  pace  of  such  commands  and 
simulator  responses  for  hard  maneuvers  where  more  sample 
points  are  needed.  The  computer  buffers  the  command  samples 
and  later  feeds  them  to  the  actual  control  system  at  the  real-time 
pace,  interpolating  between  sampled  points  as  necessary.  (This  is 
not  unlike  the  “speeding  up  on  the  straightaways  and  slowing 
down  on  the  curves”  example  previously  cited  as  an  advantage  of 
preview  control,  and  in  fact  is  what  anyone  would  do  in  making 
best  use  of  planning  time.)  The  only  requirement  is  that  the 
progression  of  planned  actions  must  keep  ahead  of  what  must  be 
delivered  “right  now”  for  real-time  control  (and  also  take  into 
account  any  time  delay). 

Disengaging  the  position  clutch  allows  one  to  move  the  simulator 
in  space  without  committing  to  later  playback,  this  for  the 
purpose  of  trying  alternative  commands  to  see  what  they  will  do. 
Disengaging  the  position  clutch  necessarily  disengages  the  time 
clutch  and  creates  a  gap  in  the  buffer  of  command  data. 
Reengaging  the  position  clutch  may  require  path  interpolation 
from  the  previous  position  by  the  actual  telerobot  controller. 

Conway  et  al.  offer  the  following  scenario  as  an  example: 
We  perform  a  complex  maneuver  with  clutches  engaged. 
We  then  disengage  the  time  clutch  to  quickly  hop  over  a 
series  of  simple  manipulation  movements,  such  as 
pushing  a  series  of  switches.  A  faint  “smoketrail” 
superimposes  the  forward  simulation  path  over  the  return 
video  display,  helping  us  to  visualize  our  progress  along 
the  chosen  path.  Having  saved  some  time,  we  then 
disengage  the  position  clutch,  and  by  trial  and  error 
movements  position  our  manipulator  in  simulation  to 
begin  a  complex  maneuver.  During  this  phase,  the 
simulation-generated  manipulator  image  moves  on  the 
display,  but  leaves  no  “smoketrail”  of  a  committed  path. 
Upon  reaching  the  correct  position  and  orientation  to 
begin  the  next  maneuver,  we  reengage  both  clutches  (the 
“smoketrail”  will  now  be  the  new  interpolated  path 
segment)  and  wait  for  the  remote  system  to  catch  up.  We 
then  begin  the  next  maneuver.  In  this  way  we  (1)  save 
some  time,  (ii)  use  the  time  saved  to  later  preposition  for 
another  action,  (iii)  avoid  taking  the  actual  system  through 
complex,  manipulatively  unnecessary  prepositioning 
movements,  and  (iv)  do  this  all  in  a  natural  way  through 
simple  controls. 

Conway  et  al.  tested  these  ideas  experimentally  using  a  Puma 
robot  arm,  a  joystick  hand  controller,  and  a  simple  two- 
dimensional  positioning  task.  They  compared  teleoperation  under 
three  conditions:  without  any  predictor  display,  with  predictor 
display,  and  with  predictor  display  plus  time  clutch.  Plots  of 
task-completion  time  as  a  function  of  task  difficulty  ratio 
(distance  moved  divided  by  diameter  of  target)  yielded  results  for 
the  first  two  conditions  which  confirmed  the  Hashimoto 
andSheridan  [68]  results  that  the  predictor  by  itself  made 
significant  improvement  (they  found  up  to  50%  shorter 
completion  times  for  some  subjects).  They  also  found  that  adding 
the  time  clutch  could  make  further  improvement  (of  up  to  40%)  if 
the  slewing  speed  of  the  robot  arm  was  constrained  to  be  very 
slow  and  if  the  operators  used  finesse  and  were  careful  not  to 
overdrive  the  system.  Various  other  researchers  have  adopted 
versions  of  the  “time  clutch.”  These  ideas  deserve  further 
development. 


Forward-backward  editing  of  commands  for 
prerecorded  manipulation 

At  the  extreme  of  time  desynchronization  is  recording  a  whole 
task  on  a  simulator,  then  sending  it  to  the  telerobot  for 
reproduction.  This  might  be  workable  when  one  is  confident  that 
the  simulation  matches  the  reality  of  the  telerobot  and  its 
environment,  or  when  small  differences  would  not  matter  (e.g., 
in  programming  telerobots  for  entertainment).  Doing  this  would 
certainly  make  it  possible  to  edit  the  robot’s  maneuvers  until  one 
was  satisfied  before  committing  them  to  the  actual  operation. 
Machida  et  al.  [76]  demonstrated  such  a  technique  by  which 
commands  from  a  master-slave  manipulator  could  be  edited  much 
as  one  edits  material  on  a  video  tape  recorder  or  a  word 
processor.  Once  a  continuous  sequence  of  movements  had  been 
recorded,  it  could  be  played  back  either  forward  or  in  reverse  at 
any  time  rate.  It  could  be  interrupted  for  overwrite  or  insert 
operations.  Their  experimental  system  also  incorporated 
computer-based  checks  for  mechanical  interference  between  the 
robot  arm  and  the  environment. 


SUPERVISORY  CONTROL 

The  basic  idea  of  supervisory  control 

The  term  supervisory  control  is  derived  from  the  close  analogy 
between  the  supervisor’s  interaction  with  subordinate  human 
staff  members  in  a  human  organization  and  a  person’s  interaction 
with  “intelligent”  automated  subsystems.  A  supervisor  of  humans 
gives  directives  that  are  understood  and  translated  into  detailed 
actions  by  staff  subordinates.  In  turn,  subordinates  collect 
detailed  information  about  results  and  present  it  in  summary  form 
to  the  supervisor,  who  must  then  infer  the  state  of  the  system  and 
make  decisions  for  further  action.  The  intelligence  of  the 
subordinates  determines  how  involved  their  supervisor  becomes 
in  the  process.  Automation  and  semi-intelligent  subsystems 
permit  the  same  sort  of  interaction  to  occur  between  a  human 
supervisor  and  the  computer-mediated  process  [1][77][78]. 

In  the  strictest  sense,  supervisory  control  means  that  one  or  more 
human  operators  are  intermittently  programming  and  continually 
receiving  information  from  a  computer  that  itself  closes  an 
autonomous  control  loop  through  artificial  effectors  and  sensors 
to  the  controlled  process  or  task  environment.  In  a  less  strict 
sense,  supervisory  control  means  that  one  or  more  human 
operators  are  continually  programming  and  receiving  information 
from  a  computer  that  interconnects  through  artificial  effectors  and 
sensors  to  the  controlled  process  or  task  environment.  In  both 
definitions  the  computer  transforms  information  from  human  to 
controlled  process  and  from  controlled  process  to  human,  but 
only  under  the  strict  definition  does  the  computer  necessarily 
close  a  control  loop  that  excludes  the  human,  thus  making  the 
computer  an  autonomous  controller  for  some  variables  at  least 
some  of  the  time. 

Figure  23  characterizes  supervisory  control  in  relation  to  the 
extremes  of  manual  control  and  full  automatic  control.  Common 
to  the  five  man-machine  system  diagrams  are  displays  and 
controls  interfaced  with  the  human  operator,  and  sensors  and 
actuators  interacting  with  a  controlled  process  or  environmental 
task.  System  1  represents  conventional  (i.e.,  not  computer-aided) 
manual  control.  In  system  2  significant  computer  transforming  or 
aiding  is  done  in  either  or  both  of  the  sensing  and  acting 
(effector)  loops.  This  corresponds  to  the  less  strict  definition  of 
supervisory  control.  Note,  however,  that  in  both  system  1  and 
system  2,  all  control  decisions  depend  upon  the  human  operator. 
If  the  human  stops,  control  stops. 

In  the  strict  forms  of  supervisory  control  (systems  3,4),  the 
human  operator  (supervisor)  programs  by  specifying  to  the 
computer  goals,  objective  tradeoffs,  physical  constraints, 
models,  plans,  and  “if-then-else”  procedures.  This  specification 
is  usually  and  most  conveniently  put  in  high-level  “natural” 
language  —  in  terms  of  desired  relative  changes  in  the  controlled 
process,  rather  than  in  terms  of  control  signals.  Once  the 
supervisor  turns  control  over  to  the  computer,  the  computer 
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Figure  23.  The  spectrum  of  control  modes.  Broken  line  indicates 
minor  loops  are  closed  through  computer,  major  loops  through 
human.  Solid  line  indicates  major  loops  are  closed  through  computer, 
minor  loops  through  human.  [From  Sheridan,  ref.  1] 


executes  its  stored  program  and  acts  on  new  information  from  its 
sensors  independently  of  the  human,  at  least  for  short  periods  of 
time.  The  human  may  remain  as  a  supervisor,  or  may  from  time 
to  time  assume  direct  control  (this  is  called  traded  control),  or 
may  act  as  supervisor  with  respect  to  control  of  some  variables 
and  direct  controller  with  respect  to  other  variables  ( shared 
control). 

Figure  24  illustrates  the  concept  of  supervisory  control.  The 
human  operator  provides  largely  symbolic  commands 
(concatenations  of  typed  symbols  or  specialized  key  presses)  to 
the  computer.  However,  some  fraction  of  her  commands  may  be 
analogic  (hand-control  movements  isomorphic  to  the  space-time- 
force  continuum  of  the  physical  task)  in  order  to  point  to  objects 
or  otherwise  demonstrate  to  the  computer  relationships  that  are 
difficult  for  the  operator  to  put  into  symbols.  The  local  or  human- 
interactive  computer  (HIC)  thus  should  be  human-friendly,  able 
to  indicate  that  it  understands  the  message  and  able  to  point  out 
that  a  specification  is  incomplete.  In  this  way  it  should  help  the 
operator  to  edit  the  message  correctly.  It  also  needs  to  interpret 
signals  from  the  distant  telerobot,  storing  and  processing  them  to 
generate  meaningful  integrated  graphic  displays.  Finally, this  local 
human-interactive  computer  should  contain  a  knowledge  base  and 
a  model  of  the  controlled  process  and  task  environment  and  be 
able  to  answer  queries  put  to  it  by  the  operator.  Meanwhile,  the 
subordinate  “remote”  or  task-interactive  computer  (TIC),  that 
accompanies  the  controlled  process  must  receive  commands, 
translate  them  into  executable  strings  of  code,  and  perform  the 
execution,  closing  each  control  loop  though  the  appropriate 
actuators  and  sensors. 


Human-Interactive  System 
(in  control  room  or  cockpit) 


multiplexed  signal  transmission 
(may  involve  bandwidth 
constraints  or  time  delays) 


Task-Interactive  System 
(remote  from  operator) 


Task  or  Controlled  Process 
(may  be  continuous  process, 
vehicle,  robot,  or  etc,) 


Figure  24.  Supervision  of  multiple  computers  and  tasks.  [From 
Sheridan,  ref.  1] 


Five  generic  supervisory  functions 

The  human  supervisor’s  functions  are:  (I)  planning  what  task  to 
do  and  how  to  do  it, 

(2)  teaching  (or  programming)  the  computer  what  was  planned; 

(3)  monitoring  the  automatic  action  to  make  sure  all  is  going  as 
planned  and  to  detect  failures;  (4)  intervening  (which  means  that 
the  supervisor  supplements  ongoing  automatic  control  activities, 
takes  over  control  entirely  after  the  desired  goal  state  has  been 
reached  satisfactorily,  or  intermpts  the  automatic  control  in 
emergencies  to  specify  a  new  goal  state  and  reprogram  a  new 
procedure);  and  (5)  learning  from  experience  so  as  to  do  better  in 
the  future.  These  are  usually  time-sequential  steps. 

Planning 

This  is  the  hardest  function  to  model.  Formally  it  means 

(1)  gaining  experience  and  understanding  of  the  physical  process 
to  be  controlled,  including  the  constraints  set  by  nature  and 
circumstances  surrounding  the  job, 

(2)  setting  goals  that  are  attainable,  or  objectives  along  with 
tradeoffs,  that  the  computer  can  “understand”  sufficiently  well  to 
give  proper  advice  or  make  control  decisions,  and 

(3)  formulating  a  strategy  for  going  from  the  initial  state  to  the 
goal  state. 

Teaching  the  computer 

The  supervisor  must  translate  goals  and  strategy  into  detailed 
instructions  to  the  computer  such  that  it  can  perform  at  least  some 
part  of  the  task  automatically,  at  least  until  the  instructions  are 
updated  or  changed  or  the  human  takes  over  by  manual  control. 
This  includes  knowing  the  requisite  command  language 
sufficiently  well  that  goals  and  instructions  can  be  communicated 
to  the  computer  in  correct  and  timely  fashion. 

Monitoring  automatic  control 

Once  the  goals  and  instructions  are  properly  communicated  to  the 
computer  for  automatic  execution  of  that  part  of  the  task,  the 
supervisor  must  observe  this  performance  to  ensure  that  it  is  done 
properly,  using  direct  viewing  or  whatever  remote  sensing 
instalments  are  available.  The  prompt  detection  of  the  presence 
and  location  of  failures,  of  conflicts  between  actions  and  goals, 
and  the  anticipation  that  either  of  these  is  about  to  occur,  are  an 
essential  part  of  the  supervisor’s  job. 

Inten>ening  to  update  instructions  or  assume  direct  control 

If  the  computer  signals  that  it  has  accomplished  its  assigned  part- 
task,  or  if  it  has  apparently  run  into  trouble  along  the  way,  the 
human  supervisor  must  step  in  to  update  instructions  to  the 
computer  or  to  take  over  control  in  direct  manual  fashion,  or 
some  combination  of  the  two.  Since  the  controlled  process  is  an 
ongoing  dynamic  system,  not  a  machine  that  can  be  arbitrarily 
stopped  and  started  again  like  a  computer,  the  takeover  itself  must 
be  smooth  so  as  not  to  cause  instability.  Similarly,  reverting  to 
the  automation  must  be  smooth. 

Learning  from  experience 

The  supervisor  must  ensure  that  appropriate  data  are  recorded  and 
computer-based  models  are  updated  so  as  to  characterize  current 
conditions  with  the  most  accurate  information.  Historical  data 
must  continuously  be  analyzed  for  trends  or  contingencies 
leading  to  abnormalities.  All  such  information  must  be  in  a  form 
usable  in  the  future  in  the  four  preceding  steps. 

Manipulating  objects  and  their  representations 

As  mankind  has  evolved  there  are  two  ways  in  which  the  human 
body  has  affected  its  environment:  by  manipulation  and  by 
communication.  Both  can  be  called  “tool  using.”  In  manipulation, 
humans  have  used  their  hands,  or  other  parts  of  the  body  or 
mechanical  tools  in  the  hands,  to  apply  forces  and  displacements 
to  physically  modify  the  environment.  In  communication,  the 
vocal  cords  or  other  parts  of  the  body  have  been  used  as  tools  to 
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make  representations  for  others  to  observe  and  understand.  As 
we  move  toward  computer  mediation  in  performing  tasks,  we 
necessarily  move  more  toward  using  our  bodies  more  for 
communication  than  for  direct  manipulation. 

The  correspondence  between  human  verbal  language  and  digital 
computer  language  has  been  widely  appreciated.  Both  are  thoughi 
of  as  alphanumeric  strings  which  have  meaning  as  a  function  of 
their  syntax  and  parsability.  What  seems  not  to  have  been  so 
widely  appreciated  is  the  close  correspondence  between  human 
verbal  language  and  human  manipulation  of  objects  in  the 
physical  environment.  This  lack  of  appreciation  is  all  the  more 
surprising  since  the  subject-verb-object  sentence  (with 
appropriate  modifiers  on  each)  corresponds  rather  directly  to  the 
hand-action-tool  “sentence,”  or  more  generally  the  logic  of 
hand/tool-action  upon-external  object.  It  has  taken  the  artifact  of 
the  teleoperator  to  make  us  aware  of  this  close  relation.  But  we 
still  do  not  have  fully  satisfactory  representational  languages  for 
manipulation. 

Symbolic  and  analogic  command  languages 

Teaching  or  programming  a  manipulation  or  mobility  task, 
including  specification  of  a  goal  state  and  a  procedure  for 
achieving  it,  and  including  necessary  constraints  and  criteria,  can 
be  formidable  or  quite  easy,  depending  upon  the  command 
hardware  and  software.  By  command  hardware  is  meant  the  way 
in  which  human  motor  action  —  hand,  foot,  or  voice  —  is 
converted  to  physical  signals  to  the  computer.  Command 
hardware  can  be  either  analogic  or  symbolic. 

Analogic  command  means  that  there  is  a  spatial  or  temporal 
isomorphism  between  human  response,  semantic  meaning, 
and/or  feedback  display.  For  example,  moving  a  control  up 
rapidly  to  increase  the  magnitude  of  a  variable  quickly,  which 
causes  a  display  indicator  to  move  up  quickly,  would  be  a  proper 
analogic  correspondence.Pointing  to  an  object  to  identify  it  is 
another  example  of  analogic  command.  Manual  force  or  position 
tracking  amounts  to  continuous  analogic  command. 

Symbolic  command,  by  contrast,  is  accomplished  by  depressing 
one  or  a  series  of  keys  (as  when  typing  words  on  a  typewriter), 
or  uttering  one  or  a  series  of  sounds  (as  in  speaking  a  sentence), 
each  of  which  has  a  distinguishable  meaning.  For  symbolic 
commands,  a  particular  series  or  concatenation  of  such  responses 
has  a  different  meaning  from  other  concatenations.  Spatial  or 
temporal  correspondence  to  the  meaning  or  the  desired  result  is 
not  requisite.  Sometimes  analogic  and  symbolic  can  be 
combined,  e.g.,  where  up-down  keys  are  both  labeled  and 
positioned  accordingly. 

It  is  natural  for  humans  to  intermix  analogic  and  symbolic 
commands  or  even  to  use  them  simultaneously.  This  happens, 
for  example,  when  one  talks  and  points  at  the  same  time,  or  plays 
the  piano  and  conducts  a  choir  with  one’s  head. 

Alphanumeric  symbols  could  also  be  used  to  instruct  the  telerobot 
to  make  a  graceful  movement,  but  they  might  better  serve  to 
communicate  knowledge-based  heuristics  for  avoiding  obstacles 
or  making  tactile  discriminations  (which  cannot  be  done  so  easily 
by  analogic  demonstration).  Symbols  might  be  better  than  analog 
demonstration  for  communicating  to  the  telerobot  how  to  draw 
given  geometric  figures  like  squares  or  triangles,  since  the  figures 
might  end  up  being  more  precise.  This  and  naming  of  objects 
pointed  to  would  be  rule-based.  Simply  typing  a  well-learned 
sequence  of  letters  or  numbers  would  be  a  skill-based  symbolic 
communication.  Table  1  illustrates  these  relationships. 

BEHAVIORAL  COMMUNICATION  CODES 

BASIS -  CONTINUOUS  ANALOGIC _ DISCRFTF  SYMBOLIC 

knowledge  play  charades  write  a  computer  program 

ru'e  point  to  a  named  object  name  an  object  pointed  to 

skill  draw  an  0  type  an  0 

Table  1.  Examples  of  symbolic  and  analogic  control  at  three 
behavioral  levels.  [From  Sheridan,  ref.  1] 


Borrowings  from  industrial  robot  command  language 

Early  industrial  robots  could  be  programmed  by  means  of  “teach 
pendants”  (small  hand-held  switch  boxes)  by  which  the  teacher, 
standing  adjacent  to  the  robot,  could  command  its  joint 
movements  one  DOF  at  a  time,  thus  programming  simple 
routines  which  could  be  refined  and  later  replayed  many  times  for 
assembly-line  operations.  More  sophisticated  teach-pendant 
command  structures  gradually  allowed  the  teaching  of  commands 
for  start,  stop,  speed,  etc.  between  various  reference  positions, 
conditional  branching  conditions,  and  arbitrarily  defined  macros 
[79].  The  early  robots  also  had  explicit  “mid-level”  symbolic 
command  languages,  such  as  Ernst’s  MH-1  [80]  and 
Unimation’s  PAL  and  HAL  [79,81].  Barber’s  MANTRAN  [82] 
was  an  early  attempt  to  build  a  higher-level  language  useful  for 
“one-of-a-kind-and-right-now”  human-to-telerobot  commands. 
Paul  [83]  reviews  the  computational  aspects  of  robot  command 
language. 

Brooks’  SUPERMAN,  and  comparison  of  alternative 
supervisory  teaching  modes 

T.  Brooks  [16]  developed  a  high-level  command  system  he  called 
SUPERMAN  which  allowed  the  supervisor  to  use  a  master  arm 
to  identify  objects  and  command  elemental  motions  in  terms  of 
those  objects.  He  showed  that  even  without  time  delay  such 
supervisory  control  —  including  both  teaching  and  execution  — 
often  took  less  time  and  had  fewer  errors  than  manual  control. 

Brooks  first  developed  software  to  enable  the  supervisor  to  teach 
the  computer  by  performing  a  manipulation  with  the  master  arm 
of  a  manipulator  and  simultaneously  code  the  objects  and  the 
movements  using  a  symbolic  keyboard.  Later,  when  the  human 
operator  required  a  particular  already-trained  manipulation,  he 
simply  “initialized”  a  new  coordinate  system  relative  to  the  old 
one  by  moving  the  teleoperator  hand  to  the  starting  point  of  the 
task  (e.g.,  grasping  a  particular  nut  or  valve  handle)  and 
signaling  for  execution  on  “this”  object.  The  computer 
automatically  retransformed  the  old  coordinates  to  a  new 
coordinate  system  and  performed  the  desired  task,  possibly  also 
following  commands  to  terminate  the  execution  at  a  previously 
identified  location  or  object.  Brooks’  supervisory  programs 
could,  upon  certain  touch  conditions’  becoming  true,  branch  into 
other  programs.  For  example,  the  telerobot  hand  could  grasp  a 
nut,  unscrew  it  half  a  turn,  pull  back  to  test  whether  it  was  off, 
and,  if  it  was,  place  it  in  a  (previously  located  by  this 
demonstration  procedure)  bucket,  or  if  it  was  not,  repeat  the 
operation. 

Six  manipulation  tasks  were  identified  for  experimental 
investigation:  retrieval  of  a  tool  from  a  rack,  returning  a  tool  to  its 
rack,  taking  a  nut  off,  grasping  an  object  and  placing  it  in  a 
container,  opening  or  closing  a  valve,  and  digging  sand  and 
putting  it  in  a  container.  In  addition,  four  manual  control  modes 
were  employed:  switch  (on-off  reverse,  or  fixed  rate),  joystick 
(variable  rate),  master-slave  position  control,  and  master-slave 
position  control  with  force  feedback.  Both  one-view  and  two- 
view  (separate  displays,  not  stereo)  video  conditions  were  tested. 
For  all  these  combinations  of  conditions  both  direct  manual 
teleoperator  and  supervisory  control  were  compared. 

In  all  cases  the  error  rates  for  supervisory  control  were  lower 
than  those  for  direct  manual  teleoperation.  Theoretically  there  is 
no  reason  why  master-slave  control  with  force  feedback  should 
be  any  faster  than  supervisory  control.  Consider  that  the 
computer  could  simply  mimic  the  human  operator’s  best 
trajectory  and  hence  be  at  least  as  fast.  Unfortunately,  in  practice 
there  is  always  a  certain  overhead  associated  with 
retransformation  of  coordinates,  trajectory  calculations,  and 
sensor  logic.  Also,  it  was  generally  observed  that  the  subjects 
were  making  adaptive  motions,  whereas  the  computer  was 
limited  to  more  rigidly  defined  trajectories  and  states. 

Thus,  even  with  no  time  delay,  supervisory  control  was  found  to 
be  more  effective  (as  determined  from  the  task  completion  times 
and  manipulation  errors)  than  switch  rate  control,  joystick  rate 
control,  and  master-slave  position  control.  Bilateral  force- 


reflecting  master-slave  control  was  found  to  be  slightly  faster 
than  supervisory  control  but  more  prone  to  errors.  Since  the 
experiments  were  performed  under  “ideal”  conditions,  it  could  be 
predicted  in  1979  that  supervisory  control  would  show  an  even 
greater  advantage  when  used  with  degraded  sensor  or  control 
loops  (time  delays,  limited  bandwidth,  etc.). 

Yoerger’s  experiments  on  teaching  modes 

Yoerger  [10]  extended  Brooks’  work,  developing  a  more 
extensive  and  robust  supervisory  command  system  that  enabled  a 
variety  of  aim-hand  motions  to  be  defined,  called  upon,  and 
combined  under  other  commands.  In  one  set  of  experiments 
Yoerger  compared  three  different  procedures  for  teaching  a  robot 
arm  to  perform  a  continuous  seam  weld  along  a  complex  curved 
workpiece.  The  end  effector  (welding  tool)  had  to  keep  1  inch 
away,  to  retain  an  orientation  perpendicular  to  the  curved  surface 
to  be  welded,  and  to  move  at  constant  speed. 

Part  of  Yoerger’s  system  was  an  on-line  computer  simulation  and 
display  that  allowed  motions  of  the  manipulator  to  be  simulated  in 
all  six  DOF  to  test  programs  before  they  were  actually  executed. 
The  operator  could  view  the  simulation  from  any  angle,  could 
translate  or  zoom  the  display,  could  run  various  simulations  in 
faster  than  real  time,  and  could  then  call  for  an  actual  execution  of 
some  already  stored  trajectory  at  some  specified  new  location. 

Yoerger  tested  his  subjects  in  three  command  (teaching)  modes 
using  the  simulation: 

Continuous  trajectory  analogic  demonstration.  The  human 
teacher  first  moved  the  master  (with  slave  following  in  master- 
slave  correspondence)  relative  to  the  workpiece  in  the  desired 
trajectory.  The  computer  would  memorize  the  trajectory,  and  then 
cause  the  slave  end-effector  to  repeat  the  trajectory  exactly. 

Discrete  point  analogic  specification  with  machine 
interpolation.  The  human  teacher  moved  the  master  (and  the 
slave)  to  each  of  a  series  of  positions,  pressing  a  key  to  identify 
each.  The  human  supervisor  would  then  key  in  additional 
information  specifying  the  parameters  of  a  curve  to  be  fitted 
through  these  points  and  the  speed  at  which  it  was  to  be 
executed,  and  the  computer  would  then  be  called  upon  for 
execution. 

Analogic  specification  of  reference  information  and 
symbolic  goal  specification  relative  to  the  reference.  In  this  mode 
the  supervisor  used  the  master-slave  manipulator  to  contact  and 
trace  along  the  workpiece,  to  provide  the  computer  with 
knowledge  of  the  location  and  orientation  of  the  surfaces  to  be 
welded.  Then,  using  the  keyboard,  the  supervisor  would  specify 
the  positions  and  orientations  of  the  end  effector  relative  to  the 
workpiece  (e.g.,  to  move  along  but  1  inch  away  from  the 
designated  surface  at  a  given  speed  and  a  given  angle).  The 
computer  could  then  execute  the  task  instructions  relative  to  the 
geometric  references  given. 

Figure  25  shows  the  average  results  for  three  experimental 
subjects,  based  on  running  measures  of  both  position  error  and 
orientation  error  in  system  performance  after  teaching  in  each  of 
the  three  modes.  Identifying  the  geometry  of  the  workpiece 
analogically,  and  then  giving  symbolic  instructions  relative  to  it, 
proved  the  constant  winner.  It  was  further  shown  that  the 
interface  decreased  the  operator’s  dependence  on  visual  feedback 
The  system  decreased  the  variability  in  performance  between 
operators,  and  the  computer  graphic  display  helped  the  operator 
understand  elements  of  the  programming  system  without 
requiring  a  formal  mathematical  description  of  how  the 
commands  work. 

Yoerger’s  results  showed  that  analogic  teaching  can  be  especially 
useful  when  an  operator  does  not  know  the  exact  coordinate 
values  of  a  position  but  can  see  via  the  graphic  or  television 
display  what  he  wants.  Using  analogic  definitions  in  combinatior 
with  symbolic  commands  simplifies  the  teaching  of  telerobotic 
tasks.  Programs  can  be  written  in  which  the  operator  points 
(moves)  to  an  object  in  the  task  environment  and  then  describes 
an  operation  to  be  done  on  the  object  by  specifying  motions  built 
on  the  previously  defined  positions. Using  the  relative 
commands,  the  arm  may  be  moved  relative  to  its  current  position 


Figure  25.  Yoerger's  results  for  alternative  methods  of  teaching. 

Mode  1,  human  demonstrated  full  trajectory,  computer  reproduced  it 
exactly.  Mode  2,  human  demonstrated  series  of  discrete  positions 
and  orientations,  computer  connected  them  with  smooth  trajectory. 
Mode  3,  human  indicated  relevant  surface  of  workpiece  and  specified 
trajectory  relative  to  it  with  symbolic  commands,  computer  executed  as 
instructed.  [From  Yoerger,  ref.  10] 


and  orientation.  Such  relative  commands  can  be  useful  for 
describing  tool  motions,  such  as  turning  a  valve  or  brushing  a 
weld. 

Mouse-based  object  level  control  (S.  Schneider  and 
R.  Cannon) 

Using  a  planar  three-DOF,  two-arm  manipulation  task,  Schneider 
and  Cannon  [84]  demonstrated  a  computer-graphic  supervisory 
command  interface  that  permitted  the  operator  to  move  a  mouse, 
click  on  an  object  to  be  moved,  drag  a  “ghost  object”  to  a  desired 
new  position,  and  then  release  the  mouse  button.  The  computer 
would  then  dutifully  perform  the  operation.  This  was  essentially 
similar  to  Brooks’  method  for  identification  by  pointing  at  the 
object  to  be  grasped  and  moved  and  to  Yoerger’s  method  of 
moving  a  computer-graphics-simulated  manipulator  to  a  point  in 
simulated  space  as  a  means  to  specify  the  subtask  goal  for 
subsequent  automatic  execution. 

Schneider  and  Cannon  added  two  modes.  In  one  the  ghost 
object,  when  in  proximity  to  a  nominal  target  (say  a  peg  in 
proximity  to  a  hole),  could  be  made  to  “snap”  into  a 
preprogrammed  relation  to  the  target  (into  the  hole).  In  another 
mode,  again  using  the  mouse,  the  operator  could  attach  one  end 
of  a  ghost  spring  to  an  arbitrary  point  on  a  ghost  object  and  the 
other  end  to  a  fixed  point  in  the  environment.  The  real  object, 
when  manipulated,  would  then  assume  the  equivalent  impedance 
characteristics.  In  general,  equivalent  stiffness,  damping,  and 
mass  can  be  added  in  this  manner  in  any  independent  direction. 

D.  Cannon’s  experiments  with  “put  that  there” 

D.  Cannon  [85]  conducted  experiments  in  analogic  task  teaching 
in  unstructured  environments  wherein  the  supervisor’s  role  was 
limited  to  task  conception  and  pointing,  and  the  telerobot  did  the 
rest.  (The  phrase  “put  that  there”  was  originally  popularized  by 
Bolt  [86]  as  a  graphics  interface  technique.)  Using  a  mobile  robot 
with  a  six-DOF  arm  and  two  pan-tilt  CCD  video  cameras 
mounted  on  a  post  (Figure  26),  the  human  operator  aimed  camera 
reticles  at  the  crucial  objects  and  destinations  to  accomplish  a  real- 
world  task.  From  camera  angle  triangulations,  in  concert  with 
combinations  of  meaningful  subphrases  such  as  “put  that.. .and 
that.. .there,”  the  telerobot  interactively  built  arm  and  mobile  base 
trajectories  for  the  immediate  or  delayed  execution  of  tasks.  The 
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Figure  26.  Cannon's  ranging  telerobot  for  "put-that-there" 
experiments.  [Courtesy  of  D.  Cannon.] 


tasks  involved  objects  and  locations  about  which  the  robot  had 
essentially  no  foreknowledge,  such  as  putting  tools  in  a  tool  box. 
items  of  trash  in  a  wastebasket,  and  blocks  on  a  pallet;  all  the 
items  had  been  strewn  randomly  in  a  workspace.  Cannon 
demonstrated  that  his  technique  reduced  human  supervisory 
control  time  and  sped  up  task-execution  time. 

Such  point-and-select  and  “ghost  object  manipulation”  methods 
are  a  bridge  between  autonomous  robotics  and  telemanipulation. 
Though  point-and-select  telerobots  with  autograsping  and 
obstacle-avoidance  systems  can  often  operate  in  unstructured 
environments  without  any  telemanipulation,  a  supervisor  may  be 
in  a  stronger  position  if  he  has  hand/gripper  telemanipulation  and 
perhaps  full  robot  telemanipulation  (up  to  the  number  of  DOF 
controllable)  at  times  during  or  at  the  ends  of  some  trajectories. 
Indeed,  an  operator  may  benefit  from  having  a  full  set  of  options 
across  the  entire  spectrum  from  autonomy  to  telemanipulation  and 
using  a  point-and-select  interface  of  some  type  to  connect  the  two 
ends  of  the  continuum.  Perhaps  this  powerful  combination  can  be 
achieved  with  very  little  overhead  if  the  hardware  and  the 
software  are  chosen  appropriately. 

In  cases  involving  significant  time  delays,  the  point-and-select 
approach  eliminates  the  need  for  the  move-and-wait  stepping 
actions  of  telemanipulation.  This  is  because  destinations,  rather 
than  incremental  motions,  are  prescribed  so  that  the  robot  can 
move  quickly  and  continuously  between  locations  of  importance. 
Task-specific  criteria,  such  as  proper  welding  speed  and  offset 
requirements,  could  be  incorporated  such  that  the  phrase  “weld 
from  there  to  there”  creates  a  routine  that  installs  a  welding  rod 
and  then,  using  proximity  sensors,  follows  the  contours  of  any 
curved  surface  between  the  two  points  while  keeping  the  angle 
and  the  spacing  of  the  rod  correct  for  welding  on  that  surface. 

The  command  “paint  all  but  that”  could  become  meaningful  with 
advanced  natural  language  systems.  In  all  such  cases,  the  role  of 
the  human  remains  at  the  highest  level  of  supervisory  control 
commensurate  with  defining  tasks  involving  objects  and 
destinations  about  which  the  robot  has  no  specific 
foreknowledge. 

Dual-mode  sensory-based  teaching  (Hirzinger  and 
Heindl) 

Hirzinger  and  Heindl  [87]  and  Hirzinger  and  Landzettel  [88] 
proposed  and  experimentally  implemented  a  technique  by  which 
the  human  operator,  either  on  or  off  line,  could  continuously 
specify  a  position  trajectory  to  a  telerobot  in  space,  using  an 
isometric  “force-joystick”  or  “sensor  ball”  to  control  rate  in  six 
axes.  Especially  in  cases  where  there  was  time  delay,  a  local 
computer  simulation  or  duplicate  hardware  teleoperator  could  be 
used  to  produce  immediate  and  easily  observable  feedback.  When 
the  telerobot  sensed  contact  with  the  environment,  or  alternatively 
when  the  operator  chose,  there  was  a  switch  to  a  force  control 
mode,  where  the  forces  applied  to  the  six  axes  of  the  sensor  ball 
served  as  reference  signals  to  a  force  a  control  loop  closed  locally 
at  the  remote  teleoperator. 

The  former  (rate)  mode  presumably  would  be  used  in  free  space, 
and  the  latter  (force)  mode  when  contact  with  external  objects 


threatens  instability  and/or  the  operator  needs  force  feedback 
(e.g.,  in  fitting  a  peg  in  a  hole).  Note  that  Hirzinger’s  system  was 
not  true  force  feedback.  The  spring  restoring  forces  from  the  six- 
axis  joystick  were  what  was  felt  by  the  operator,  and  these  were 
also  giving  reference  signals  to  the  local  force  control  loops. 
Ranging  sensors  attached  to  the  end  effector  could  be  made  to  act 
as  pseudo-force  sensors  in  contributing  to  the  balance  of  end- 
effector  forces  to  commanded  forces. 

Asada-Yang  techniques  for  teaching  force-position- 
time  relations  by  demonstration 

Asada  and  colleagues  have  experimented  with  a  variety  of 
methods  for  teaching  by  demonstrating.  Asada  and  Yang  [89] 
demonstrated  a  system  to  capture  the  deburring  skill  of  an 
experienced  machinist  and  transfer  that  skill  to  a  robot.  The 
machinist  taught  the  computer  by  repeated  demonstrations,  and  at 
the  same  time  a  variety  of  sensor  signals  were  recorded,  such  as 
forces,  positions,  grinding-wheel  speed  and  torque,  and  sounds. 
Analysis  of  these  data  by  means  of  discriminant  functions 
determined  an  “average”  mapping  or  control  law  from  process 
state  variables  (inputs  to  the  human)  into  control  actions  (human 
outputs). 

The  teaching  process  consisted  of 

(1)  predetermining  what  single  “feature”  characterizes  the  data  as 
read  by  each  of  the  L  sensors, 

(2)  transforming  the  data  from  each  teaching  run  for  each  action 
A;  of  N  such  control  actions  into  a  vector  x  in  “feature  space” 
whose  elements  are  feature  weightings, 

(3)  normalizing  each  axis  of  feature  space  by  the  standard 
deviation  of  all  N  blocks  of  data  (one  for  each  of  N  actions),  and 

(4)  computing  the  mean  and  the  standard  deviation  in  feature 
space  of  all  the  vectors  corresponding  to  each  Ag 

The  class  wj  was  then  defined  by  a  hypersphere  in  feature  space 
centered  on  that  mean  and  having  a  radius  equal  to  that  standard 
deviation.  Later,  when  running  the  program,  whatever  x' 
occurred  was  then  compared  with  all  N  hyperspheres  in  feature 
space,  and  the  action  Aj  corresponding  to  the  closest  one  (by 
mean  square  distance)  was  triggered.  Asada  and  Liu  (1990) 
proposed  similar  teaching  by  means  of  neural-net  conditioning. 

Funda’s  scheme  for  recoding  operator  actions  into 
commands  by  class  recognition 

Funda  et  ai.  [90  developed  a  system  wherein  the  operator 
programs  by  kinesthetic  as  well  as  visual  interactions  with  a 
(virtual)  computer  simulation.  The  instructions  to  be 
communicated  to  the  actual  telerobot  are  generated  automatically 
in  a  more  compact  form  than  record  and  playback  of  analog 
signals.  Several  free-space  motions  and  several  contact,  sliding 
and  pivoting  motions,  which  constitute  the  terms  of  the  language, 
are  generated  by  automatic  parsing  and  inteipreting  of  kinesthetic 
command  strings  relative  to  the  model.  These  are  then  sent  on  as 
instruction  packets  to  the  remote  slave.  The  Funda  et  al. 
technique  also  provides  for  error  handling.  When  errors  in 
execution  are  detected  at  the  slave  site  (e.g.,  because  of  operator 
error,  discrepancies  between  the  model  and  real  situation  and/or 
the  coarseness  of  command  reticulation),  information  is  sent  back 
to  help  update  the  simulation.  This  is  to  represent  the  error 
condition  to  the  operator  and  allow  him  to  more  easily  see  and 
feel  what  to  do  to  correct  the  situation. 

Action,  direction,  agreement,  negation,  and  delegation 
as  bases  for  command 

T.  Brooks  [91]  reports  on  experiments  with  five  progressively 
graded  levels  of  human  supervisory  control  of  a  telerobotic 
vehicle,  which  he  calls  action,  direction,  agreement,  negation, 
and  delegation.  By  action  he  means  either  real-time  direct  and 
continuous  control  by  the  operator  or  continuous  record  and  then 
playback.  In  either  case  the  operator  must  “do”  each  and  every 
step  of  the  task.  Direction  means  that  the  operator  specifies  each 
in  a  series  of  small  incremental  goals,  and  the  computer  interprets 
and  executes  these  one  step  at  a  time.  Agreement  means  that  the 
computer  selects  an  action  but  waits  for  the  operator  to  agree  to  it; 
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if  he  doesn’t,  another  action  can  be  selected,  and  so  on.  Negation 
means  that  the  computer  seeks  to  carry  out  a  task  autonomously 
but  the  human  operator  may  override  it.  Delegation  means  that  the 
human  operator  specifies  overall  goals,  then  turns  over  part  or  all 
of  a  task  to  the  computer  to  perform  as  it  sees  fit.  The  computer 
in  this  case  has  no  responsibility  to  inform  the  operator  what  it 
decides.  At  the  time  of  this  writing  the  implementation  had  not 
been  completed. 

Communication  confirmation  in  command  and  display 

It  is  easy  to  think  that  communication  with  a  machine  requires 
only  giving  action  commands  and  getting  back  indications  of 
state,  where  both  command  and  state  information  are  coded 
symbolically,  analogically,  or  in  combination.  Indeed,  that  has 
been  characteristic  of  most  machines,  from  dishwashers  to 
automobiles  to  aircraft.  However,  reflection  on  how  humans 
communicate  with  one  another  reveals  that  something  more  is 
going  on,  something  very  important:  there  is  intermediate 
feedback  for  both  giving  commands  and  getting  state 
information,  as  shown  in  Figure  27.  If  an  assignment  is  given  to 
an  unsophisticated  subordinate  he  may  simply  go  off  and  act  as 
best  he  understands,  much  as  a  simple  machine  would.  A  more 
sophisticated  human  worker  (or  an  intelligent  machine)  would 
respond  at  that  point  with  something  like  “This  is  what  I 
understand  you  have  asked  me  to  do;  is  that  correct?’’,  leaving  the 
supervisor  the  opportunity  to  clarify,  and  not  have  to  guess 
whether  the  subordinate  has  understood.  Similarly,  when  one 
person  reports  on  the  state  of  objects  or  events,  the  intelligent 
listener  is  likely  to  nod  or  otherwise  indicate  what  he 
understands,  or  to  ask  for  clarification.  The  human  supervisor  of 
an  intelligent  machine  should  be  given  the  opportunity  to  do  the 
same,  with  the  machine  then  confirming,  clarifying,  or  providing 
more  detail  as  necessary.  This  is  the  essence  of  human  dialog, 
and  it  should  be  built  into  supervisory  control  systems. 


FUNCTION  OF  HUMAN  FORM  OF  FUNCTION  OF  COMPUTER 
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Figure  27.  intermediate  feedback  in  command  and  display.  Heavy 
arrows  indicate  the  conventional  understanding  of  functions.  Light 
arrows  indicate  critical  additional  functions  which  tend  to  be 
neglected.  [From  Sheridan,  ref.  1] 


Voice  command 

Voice-recognition  systems  (for  voice  command  by  human 
operators)  are  now  widely  available  at  reasonable  prices.  They  all 
must  be  trained  by  one  or  more  speakers  on  a  limited  vocabulary. 
Naturally  there  are  tradeoffs  involving  size  of  vocabulary, 
speaker  training,  speaking  speed  and  style,  number  of  different 
speakers  to  be  accommodated  by  the  same  algorithm  (one  speaker 
is  always  preferred),  recognition  reliability,  and  cost.  If  speech 
consists  of  disconnected  words,  recognition  is  much  more 
straightforward  than  if  words  are  connected  as  in  natural  speech. 
Further,  a  single  speaker  will  modify  the  sound  of  his  or  her 
voice  as  a  function  of  stress,  fatigue,  and  attention. 


Speech-recognition  algorithms  are  of  various  sorts,  ranging  from 
rule-based  techniques  and  template  matching  more  probabilistic 
or  self-learning  approaches  such  as  hidden  Markov  modeling  and 
neural  networks.  Probabilistic  algorithms  generally  store 
contingent  probabilities  based  on  the  two  previous  words 
spoken.  Measures  of  the  vocabulary  such  as  perplexity,  defined 
as  2(entroPy),  or  an  equivalent  number  of  equally  likely  events  to 
be  distinguished,  are  commonly  used. 

It  is  important  that  the  designer  of  speech  command  formats 
consider  not  only  speech-recognition  reliability  and  speed,  but 
also  the  acceptability  and  naturalness  of  use  by  the  speaker.  For 
example,  moving  a  cursor  by  using  the  speech  commands  “Up, 
up,  up,  left,  left”  would  be  easy  enough  for  the  computer  but 
time-consuming  for  the  operator,  whereas  “Go  to  the  top  right 
corner  of  the  rectangle”  should  be  understandable  by  the 
computer  and  much  more  natural  for  the  operator. 

Voice  command  for  teleoperation  has  been  experimented  with  for 
at  least  a  decade  [92],  but  thus  far  its  acceptance  has  been  limited. 

Computer-graphic  aids  for  avoiding  obstacles 

Park  [93]  developed  a  computer-graphic  control  aid  by  which  a 
supervisor  could  plan  and  essentially  try  out  an  anticipated 
telerobot  arm  movement  in  a  simulation,  again  before  committing 
to  the  actual  move.  He  assumed  that  for  some  obstacles  the 
positions  and  orientations  were  already  known  and  represented  in 
a  computer  model.  The  user  commanded  each  straight-line  move 
to  a  subgoal  point  in  three-space  by  designating  a  point  on  the 
floor  or  the  lowest  horizontal  surface  (such  as  a  table  top)  by 
moving  a  cursor  to  that  point  (say  A  in  figure  28a)  and  clicking, 
then  lifting  the  cursor  by  an  amount  corresponding  to  the  desired 
height  of  the  subgoal  point  (say  A)  above  that  floor  point  and 
observing  on  the  graphic  model  a  blue  vertical  line  being 
generated  from  the  floor  point  to  the  subgoal  point  in  space.  This 
process  was  repeated  for  successive  subgoal  points  (say  B  and 
C).  The  user  could  view  the  resulting  trajectory  model  (figure 
28b)  from  any  desired  perspective  (though  the  “real”  environment 
could  be  viewed  only  from  the  perspective  provided  by  the  video 
camera’s  location). 

Either  of  two  collision-avoidance  algorithms  could  be  invoked:  a 
detection  algorithm  which  indicted  where  on  some  object  a 
collision  occurred  as  the  arm  was  moved  from  one  point  to 
another,  or  an  automatic  avoidance  algorithm  which  found  (and 
drew  on  the  computer  screen)  a  minimum-length  no-collision 
trajectory  from  the  starting  point  to  the  new  subgoal  point.  Park’s 
aiding  scheme  also  allowed  new  observed  objects  to  be  added  to 
the  model  by  graphically  “flying”  them  into  geometric 
correspondence  with  the  model  display.  Another  aid  was  to 
generate  “virtual  objects”  for  any  portion  of  the  environment  in 
the  umbral  region  (not  visible)  after  two  video  views  (figure 
28c).  In  this  case  the  virtual  objects  were  treated  in  the  same  way 
in  the  model  and  in  the  collision-avoidance  algorithms  as  the 
visible  objects.  Experiments  with  this  technique  showed  that  it 
was  easy  to  use  and  that  it  improved  safety  greatly. 

Chiruvolu  [94]  experimented  with  the  idea  of  superposing  on  the 
video  model-based  images  allowing  the  operator  to  “see  through” 
the  teleoperator  arm  and  hand.  This  allows  for  peg-in-hole  or 
other  assembly  operations  which  otherwise  are  not  possible  with 
limited  camera  views. 


COGNITION  AND  MENTAL  MODELS 

Mental  models  in  relation  to  other  “internal”  models  in 
teleoperator  systems 

Before  elaborating  on  the  relation  of  mental  models  and  decision 
aids  to  the  plan-teach-monitor-intervene-learn  functions,  it  is 
important  to  consider  what  other  kinds  of  models  are  embodied 
within  a  supervisory  control  system,  and  where,  physically, 
these  are  located.  Figure  29  shows  four  loci  of  models  in  a 
telerobot  system:  (1)  mental  models  (presumably  resident  in  the 
supervisor’s  head),  (2)  software-based  models  in  the  computer. 
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Figure  28.  Park’s  display  of  computer  aid  for  obstacle  avoidance,  a, 
human  specification  of  subgoal  points  on  graphic  model;  b,  computer 
display  of  composite  planned  trajectory  with  lines  to  floor  to  indicate 
heights;  c,  generation  of  virtual  obstacles  for  single  viewing  position 
(above)  and  pair  of  viewing  positions  (below).  [From  Park,  ref.  93]] 


3.  COMPUTER 


Figure  29.  Four  loci  of  models  in  the  supervisory  control  system:  (1) 
mental  model;  (2)  display  representation;  (3)  control  configuration;  (4) 
computer  model.  [Courtesy  of  W.  Verplank.] 


(3)  representations  of  the  telerobot-task  in  the  configuration  of  the 
human’s  hand  controls,  and  (4)  representations  of  the  telerobot- 
task  in  the  graphics-text  presentation  on  the  supervisor’s  displays 
(including  computer-generated,  video,  and  other-than-visual 
displays).  The  latter  two  are  not  typically  considered  models,  but 
they  truly  are.  The  arrangement  of  what  the  operator  sees  and 
what  he  does  with  her  hands  and  how  this  corresponds  with  what 
he  thinks  is  critical.  Miscorrespondence  (inability  or  difficulty  in 
mapping  from  eye  or  other  sensor  to  mind,  and  then  to  hand)  is 
likely  to  cause  delays  and  errors.  The  computer’s  internal  models 
will  ran  at  their  own  speed  and  in  terms  of  whatever  the 
programmer  decided,  but  these  must  bear  correspondence  to  the 
mental  and  display-control  models.  All,  of  course,  must 
correspond  to  the  external  reality.  Designing  in  this 
correspondence  is  not  a  trivial  matter.  In  the  parlance  of 
conventional  human  factors,  notions  of  display-control 
compatibility,  user  expectation  stereotypes,  naturalness,  and 
transparency  are  often  used. 

Some  modeling  challenges 

Modeling  the  human  versus  modeling  the  human-machine 
combination 

One  of  the  most  interesting  modeling  questions  is  “What  is 
modeled?”.  The  kinematics  and  dynamics  of  mechanical 
machines,  and  the  transfer  functions  and  logical  mappings  of 
electronic  systems  of  a  teleoperator  or  telerobot,  may  be  modeled 
in  usual  ways.  When  the  human  operator  is  included,  however, 
there  is  a  challenging  problem.  It  has  been  shown  [26]  in  simple 
manual  control  systems  that  the  best-fit  (in  a  least-squares  sense) 
dynamic  model  of  human  input-output  characteristics  changes 
dramatically  as  a  function  of  the  controlled  process.  In  fact, 
within  a  range  of  controlled  processes  within  which  the  human 
operator  can  sucessfully  maintain  control,  the  human  tends  to 
adopt  the  inverse  dynamics  of  the  controlled  process,  such  as  to 
keep  the  open  loop  characteristic  of  human-plus-controlled- 
process  constant  [26]  and  in  a  form  as  close  as  possible  to 
optimal  within  the  fixed  neuromuscular  and  neural  processing 
constrains  of  the  human. 

There  is  every  reason  to  believe  that  humans  attempt  to  do  the 
same  for  more  complex  systems,  yet  our  understanding  of  this 
seeming  invariant  property  of  human-machine  systems  remains 
poorly  tested  or  understood  beyond  the  most  simple  linear  control 
loops  [1], 

Describing  system  performance 

How  to  describe  in  an  objective  way  the  performance  of  human 
and  teleoperator  in  performance  of  tasks?  We  have  available 
neither  the  descriptive  language  nor  the  theory  to  do  the  task 
properly.  We  need  a  language  to  describe  the  repositioning  in 
time  and  space  and  force  of  the  salient  elements  of  the 
teleoperator  and  the  objects  in  the  environment  in  interaction. 
Continuous  trajectories  in  a  space  of  6  (DOF)  times  the  number 
of  independent  elements  in  the  teleoperator  plus  environment 
would  descrive  everything,  but  this  clearly  is  infeasible.  Some 
more  compact  description  is  obviously  preferable.  For  example, 
a  musical  score  describes  what  a  symphony  orchestra  does  in  a 
way  which  is  more  or  less  repeatable.  There  are  even  ways  of 
scoring  dance.  Yet  currently  we  have  no  acceptable  way  of 
"scoring"  a  telemanipulation  task. 

Modeling  direct  and  supervisory  human  control 

By  direct  control  we  mean  the  human  takes  what  is  given 
(observed  by  him  or  her)  and  directly  (continuously  of 
intermittently)  operates  controls  which  cause  movements  of  the 
teleoperator  in  one-to-one  mapping.  Master-slave  position 
control,  joy-stick  rate  control,  and  switch-operated  control  for 
each  degree  of  freedom  are  examples.  A  model  of  human  control 
in  this  case  can  be  made  by  equations,  by  an  input-output  table, 
by  a  state-directed  graph,  etc. 

Modeling  supervisory  control  is  different,  and  mush  more 
difficult.  The  off-line  and  heavily  mental  events  of  planning. 
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teaching,  monitoring,  intervening  and  learning  are  in  stark 
contrast  to  sensory-motor  skill,  and  are  sufficiently  decoupled 
from  the  instantanous  phyical  movements  of  the  teleoperator 
mechanism  that  the  above  forms  of  model  are  inadequate. 
Furthermore  the  very  essence  of  supervision  is  that  that  human  is 
proactive  rather  than  reactive,  determining  what  task  is  to  be 
done,  and  how  it  is  to  be  done.  In  fact,  one  must  attribute  to  the 
human  supervisor  some  measure  of  free  will  (which  normally 
does  not  come  up  in  characterizing  the  task  of  a  subordinate 
worker).  Free  will,  by  no  defnition  is  easy  to  model. 
Furthermore,  assuming  the  supervisor  is  using  some  form(s)  of 
decision-aiding,  there  is  the  question  of  whether  the  human  is 
slavishly  folowing  the  decision  aid,  or  whether  the  human  is 
thinking  and  deciding  independedntly  of  the  decision  aid.  Of 
course,  if  it  is  the  human-plus-computer  in  tandem  that  one  is 
modeling  (as  per  the  discussion  above)  then  a  model  may  not 
have  to  discriminate  between  human  and  computer.  Finally,  one 
can  assert  that  modeling  the  human  supervisor  is  exacerbated  by 
the  sheer  complexity  of  the  kinds  of  systems  likely  to  be 
encountered. 

Modeling  mental  state:  subjective  workload  and  sense  of  presence 

Until  recent  years  no  one  much  worried  about  mental  states  of  the 
human  operator  which  might  relate  to  performance.  Nothing 
seemed  to  count  but  the  performance  of  the  system,  and  operators 
were  selected  and  trained  and  given  the  displays  and  controls  to 
enable  them  to  do  the  best  job  possible,  quite  apart  from  how 
they  felt  subjectively.  Gradually  however,  it  became  clear  that 
mental  state  affects  physical  performance,  and  if  one  could 
measure  and  predict  mental  state,  then  perhaps  one  could  predict 
performance. 

Mental  state  is  not  directly  and  physically  measurable,  and 
historically  some  psychologists  (traditionally  the  “behaviorists’ ) 
have  asserted  that  even  attampts  to  infer  mental  state  are  fruitless. 
However,  there  is  much  current  interest  in  mental  state,  and  there 
are  several  examples  of  the  attributes  of  mental  state  in  which 
there  is  currently  keen  interest. 

One  of  these  is  mental  workload.  The  idea  is  that  if  mental 
workload  is  excessive,  it  is  a  precusor  to  a  sudden  and 
catastrophic  decrement  in  human  performance,  though  by 
measuring  performance  directly  one  cannot  usually  anticipate  that 
performance  is  about  to  drop  off.  Mental  workload  has  been 
measured  [96]  for  airline  pilots  and  nuclear  power  plant 
operators,  and  it  is  appropriate  to  measure  it  for  human  operators 
of  teleoperators  performing  prolonged  activities  in  hazardous 
waste  removal,  radioactive  or  chemically  toxic  laboratory  tasks, 
etc. 
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Topics  discussed  in  this  volume  include  pattern 
recognition  in  computer  vision;  computer  vision; 
sampling  and  coding  in  computer  vision;  image 
processing;  morphological,  wavelet,  and  Gabor 
processing;  and  neural  nets  and  fuzzy  logic  in 
machine  vision.  Papers  are  presented  on  real-time 
model-based  vision  for  industrial  domains,  a  vision 
system  based  on  a  classification  connectionist  algo¬ 
rithm,  finding  landmark  features  under  a  broad 
range  of  lighting  conditions,  a  high-speed  low- 
latency  portable  visual-sensing  system,  image  analy¬ 
sis  of  photochromic  ink  for  securing  applications, 
and  image  coding  and  image  activity  measurement. 
Attention  is  also  given  to  the  acquisition  of  range 
images  in  an  integrated  vision  system  environment, 
a  robust  line  extraction  and  matching  algorithm,  a 
new  entropy  operator  for  edge  abstraction,  wavelet 
image  representation  and  applications  in  computer 
vision,  morphological  granulometric  shape  recogni¬ 
tion  in  noise,  adaptation  of  Gabor  filters  for  simula¬ 
tion  of  human  preattentive  mechanism  for  a  mobile 
robot,  a  general  learning  scheme  for  robot  coordi¬ 
nate  transformations  using  dynamic  neural  network, 
and  the  design  and  implementation  of  a  fuzzy  logic 
yaw  controller. 

Sensorbased  space  robotics  -  ROTEX  and  its 
telerobotic  features 

94A11283 

HIRZINGER,  G.;  LANDZETTEL,  K.;  DIETRICH, 
J.  (DLR,  Oberpfaffenhofen,  Germany)  IAF,  Interna¬ 
tional  Astronautical  Congress,  44th,  Graz,  Austria, 
Oct.  16-22,  1993.  21  p. 

In  early  1993,  the  space  robot  technology  experi¬ 
ment  ROTEX  has  flown  with  Space  Shuttle  CO¬ 
LUMBIA  (Spacelab  mission  D2  on  flight  STS-55 
from  April  26  to  May  6).  A  multisensory  robot  on 
board  the  spacecraft  successfully  worked  in  autono¬ 
mous  modes,  teleoperated  by  astronauts,  as  well  as 
in  different  telerobotic  ground-control  modes.  These 
include  on-line  teleoperation  and  telesensor  pro¬ 
gramming,  a  task-level  oriented  programming  tech¬ 
nique  involving  'learning  by  showing'  concepts  in  a 
virtual  environment.  The  robot's  key  features  are  its 
multisensory  gripper  and  the  local  sensory  feedback 
schemes,  which  are  the  basis  for  shared  autonomy. 
The  corresponding  man-machine  interface  concepts 
using  a  6  DOF  nonforce-reflecting  control  ball  and 
visual  feedback  to  the  human  operator  are  ex¬ 
plained.  Stereographic  simulation  on-ground  was 
used  to  predict  the  robot's  free  motion  and  the 
sensor-based  path  refinement  on  board;  prototype 
tasks  performed  by  this  space  robot  were  the  assem¬ 
bly  of  a  truss  structure,  connecting/disconnecting  an 


electrical  plug  (orbit  replaceable  unit  exchange),  and 
grasping  free-floating  objects. 

On  the  nonlinear  dynamics  of  a  space  platform 
based  mobile  flexible  manipulator 
94A11110 

MODI,  V.  J.;  MAH,  H.  W.;  MISRA,  A.  K.  (British 
Columbia  Univ.,  Vancouver,  Canada);  (McGill 
Univ.,  Montreal,  Canada)  IAF,  International  Astro- 
nautical  Congress,  44th,  Graz,  Austria,  Oct.  16-22, 
1993.  22  p.  Research  supported  by  Centers  of  Ex¬ 
cellence  Program. 

A  relatively  general  formulation  is  developed  for 
studying  the  dynamics  of  an  orbiting  arbitrary  chain 
of  translating,  slewing  flexible  bodies.  The  formula¬ 
tion  accounts  for  transverse,  axial,  and  torsional 
deformation  of  beams.  The  model  takes  into  ac¬ 
count  joint  flexibility  in  three  dimensions  as  well  as 
specified  and  generalized  coordinates  at  the  joints, 
with  freedom  to  transverse  over  a  flexible  platform 
free  to  librate  and  carrying  a  flexible  payload.  The 
model  can  also  analyze  a  cluster  of  flexible  bodies 
at  joints  forming  'flower  petal-type’  configurations, 
rigid  central-body-based  geometry  applicable  to  a 
large  class  of  scientific  and  communications  satel¬ 
lites.  The  versatility  of  the  formulation  permits 
dynamical  analysis  and  nonlinear  control  of  a  wide 
class  of  space-  and  ground-based  manipulators. 

Optimal  trajectories  for  a  manipulator  on  a  satellite 
by  the  method  of  dynamic  programming  (DP) 

93A56261 

YAMAGUCHI,  ISAO;  KIDA,  TAKASHI;  UENO, 
SEIYA;  TANAKA,  MASAKI  (National  Aerospace 
Lab.,  Chofu,  Japan);  (Yokohama  National  Univ., 
Japan);  (Toshiba  Corp.,  Kawasaki,  Japan)  In:  Space 
Sciences  and  Technology  Conference,  35th, 
Nagaoka,  Japan,  Oct.  28-31,  1991,  Proceedings 
(A93-56251  24-12).  Tokyo,  Japan  Society  for  As¬ 
tronautical  and  Space  Sciences,  1991,  p.  29,  30. 

This  paper  discusses  the  optimal  trajectories  genera¬ 
tion  for  a  space  robot  manipulator  by  the  method  of 
dynamic  programming  (DP).  The  results  of  numeri¬ 
cal  examples  in  which  the  attitude  error  for  the 
two-dimensional  model  with  a  two-joint  arm  is 
minimized  are  in  good  agreement  with  the  analyti¬ 
cal  solutions. 

Skill  compensation  and  dynamic  coupling  of 
macro/smart  effector  system 

93A56260 

MACHIDA,  KAZUO;  TODA,  YOSHITSUGU; 
IWATA,  TOSHIAKI  (Electrotechnical  Lab., 
Tsukuba,  Japan)  In:  Space  Sciences  and  Technology 
Conference,  35th,  Nagaoka,  Japan,  Oct.  28-31, 

1991,  Proceedings  (A93-56251  24-12).  Tokyo,  Ja¬ 
pan  Society  for  Astronautical  and  Space  Sciences, 
1991,  p.  25,  26. 

A  smart  end  effector  was  developed  to  add  dexter¬ 
ous  and  flexible  capability  to  a  long  space  manipu¬ 
lator  arm.  It  provides  fine  adjustment  for  precise 
error  compensation  and  delicate  force  control  by  a 
remote-end  sensor  feedback.  The  performance  of  the 
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skill  compensation  and  the  dynamic  coupling  prob¬ 
lem  between  the  long  arm  and  the  smart  end  effec¬ 
tor  are  examined. 

Control  of  a  spacerobot  at  the  time  of  capturing  a 
target 

93A56259 

YAMADA,  KATSUHIKO;  YOSHIKAWA,  SHOJI; 
KOYAMA,  HIROSHI  (Mitsubishi  Electric  Corp., 
Amagasaki,  Japan)  In:  Space  Sciences  and  Technol¬ 
ogy  Conference,  35th,  Nagaoka,  Japan,  Oct.  28-31, 
1991,  Proceedings  (A93-56251  24-12).  Tokyo,  Ja¬ 
pan  Society  for  Astronautical  and  Space  Sciences, 
1991,  p.  21-24. 

When  a  spacerobot  captures  a  target,  impact  forces 
exert  on  the  hand  and  on  the  target.  We  studied  the 
relation  between  the  velocity  of  the  hand  to  the 
target  and  the  magnitude  of  the  impact  forces.  The 
magnitude  can  be  estimated  by  the  eigenvalues  of 
the  inverse  inertia  matrix  of  the  spacerobot  and  the 
target. 

On  the  capture  control  for  space  robot 

93A56257 

FUJII,  HIRONORI;  MURAYAMA,  TSUTOMU 
(Tokyo  Metropolitan  Inst,  of  Technology,  Japan)  In: 
Space  Sciences  and  Technology  Conference,  35th, 
Nagaoka,  Japan,  Oct.  28-31,  1991,  Proceedings 
(A93-56251  24-12).  Tokyo,  Japan  Society  for  As¬ 
tronautical  and  Space  Sciences,  1991,  p.  17,  18. 

A  capture  control  is  studied  in  application  for  space 
robot  technology.  The  acceleration  control  is  one  of 
the  widely  used  methods  for  this  kind  of  problem. 

In  that  control  law,  the  feedback  gains  for  position 
and  velocity  are  set  to  be  constant  with  respect  to 
the  time.  In  the  present  study,  the  feedback  gains 
are  treated  as  functions  of  state  variables  and  time, 
and  thus  the  responses  of  the  system  can  be  shaped 
more  freely.  The  controlled  system  is  also  assured 
its  stability  in  the  sense  of  Liapunov. 

Research  of  a  free-flying  telerobot.  V  -  Handling  a 
target  with  multi-arms 

93A56255 

TODA,  YOSHITUGU;  IWATA,  TOSHIAKI; 
MACHIDA,  KAZUO;  ASAKURA,  MAKOTO; 
FUKUDA,  YASUSI;  OOTUKA,  AKIKO 
(Electrotechnical  Lab.,  Tsukuba,  Japan);  (Toshiba 
Corp.,  Kawasaki,  Japan)  In:  Space  Sciences  and 
Technology  Conference,  35th,  Nagaoka,  Japan,  Oct. 
28-31,  1991,  Proceedings  (A93-56251  24-12).  To¬ 
kyo,  Japan  Society  for  Astronautical  and  Space 
Sciences,  1991,  p.  11,  12. 

This  report  presents  manipulation  and  handling 
topics  of  an  ongoing  program  for  a  research  and 
development  of  a  free-flying  telerobot  for  space  use. 
A  developed  ground  experiment  model  has  two 
manipulators  and  a  capturing  mechanism.  A  sensory 
feed  back  control  method  enables  impedance  and 
active  limp  control  of  manipulators.  We  conclude 
that  these  control  methods  are  effective  when  the 
telerobot  catches  a  target  with  a  manipulator,  moves 
with  a  manipulator  or  manipulators,  transfers  one 


manipulator  to  another,  or  transfers  manipulators  to 
a  capturing  mechanism. 

Development  of  an  autonomous  satellite  robot  for 
retrieving  a  satellite 

93A56253 

KOMATSU,  TADASHI;  UENOHARA, 

MICHIHIRO;  IIKURA,  SHOICHI;  MIURA, 
HIROFUMI;  SHIMOYAMA,  ISAO  (Toshiba  Corp., 
Kawasaki,  Japan);  (Tokyo  Univ.,  Japan)  In:  Space 
Sciences  and  Technology  Conference,  35th, 

Nagaoka,  Japan,  Oct.  28-31,  1991,  Proceedings 
(A93-56251  24-12).  Tokyo,  Japan  Society  for  As¬ 
tronautical  and  Space  Sciences,  1991,  p.  7,  8. 

We  developed  a  two-dimensional  operation  test-bed 
of  an  autonomous  free-flying  space  robot  for  re¬ 
trieving  a  satellite.  This  robot  is  floating  on  a  planar 
base  using  air  bearings  and  is  able  to  fly  around 
using  thrusters  for  position  control  and  a  control 
moment  gyro  for  attitude  control.  This  robot  also 
installs  hardware  systems  such  as  vision  systems, 
board  computers,  image  processing  units,  and  soft¬ 
ware  systems  such  as  algorithms  for  path  plannings. 

Remote  surface  inspection  system 

93A55469 

HAY  ATI,  SAMAD;  BALARAM,  J.;  SERAJI, 
HOMAYOUN;  KIM,  WON  S.;  TSO,  KAM  S. 

(JPL,  Pasadena,  CA);  (SoHar  Corp.,  Beverly  Hills, 
CA)  Robotics  and  Autonomous  Systems  (ISSN 
0921-8890),  vol.  11,  no.  1,  May  1993,  p.  45-59. 
(Previously  announced  in  STAR  as  N93-32099) 

This  paper  reports  on  an  on-going  research  and 
development  effort  in  remote  surface  inspection  of 
space  platforms  such  as  the  Space  Station  Freedom 
(SSF).  It  describes  the  space  environment  and  iden¬ 
tifies  the  types  of  damage  for  which  to  search.  This 
paper  provides  an  overview  of  the  Remote  Surface 
Inspection  System  that  was  developed  to  conduct 
proof-of-concept  demonstrations  and  to  perform 
experiments  in  a  laboratory  environment.  Specifi¬ 
cally,  the  paper  describes  three  technology  areas:  (1) 
manipulator  control  for  sensor  placement;  (2)  auto¬ 
mated  non-contact  inspection  to  detect  and  classify 
flaws;  and  (3)  an  operator  interface  to  command  the 
system  interactively  and  receive  raw  or  processed 
sensor  data.  Initial  findings  for  the  automated  and 
human  visual  inspection  tests  are  reported. 

A  controller  design  framework  for  telerobotic 

systems 

93A54269 
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IEEE  Transactions  on  Control  Systems  Technology 
(ISSN  1063-6536),  vol.  1,  no.  1,  March  1993, 
p.  50-62. 

This  paper  presents  a  framework  for  designing  a 
telerobotic  system  controller.  This  controller  is  de¬ 
signed  so  the  dynamic  behaviors  of  the  master  robot 
and  the  slave  robot  are  functions  of  each  other.  This 
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paper  first  describes  these  functions,  which  the 
designer  chooses  based  upon  the  application,  and 
then  proposes  a  control  architecture  to  achieve  these 
functions.  To  guarantee  that  the  specified  functions 
and  proposed  architecture  govern  the  system  behav¬ 
ior,  H(infinity)  control  theory  and  model  reduction 
techniques  are  used.  Several  experiments  were  con¬ 
ducted  to  verify  the  theoretical  derivations.  This 
control  method  is  unique,  because  it  does  not  re¬ 
quire  any  transfer  of  either  position  or  velocity 
information  between  the  master  robot  and  the  slave 
robot;  it  only  requires  the  transfer  of  forces.  Al¬ 
though  this  property  leads  to  a  wider  communica¬ 
tion  bandwidth  between  the  master  and  slave  robots, 
the  entire  system  may  still  suffer  from  a  positional 
error  buildup  between  the  master  robot  and  slave 
robot. 

Data  integration  in  multi-sensor  based  robotic 
workstations 

93A54229 

CHEN,  QIN;  LUH,  J.  Y.  S.  (Clemson  Univ.,  SC) 

In:  IEEE  International  Conference  on  Systems  Engi¬ 
neering,  Kobe,  Japan,  Sept.  17-19,  1992,  Proceed¬ 
ings  (A93-54226  23-66).  New  York,  Institute  of 
Electrical  and  Electronics  Engineers,  Inc.,  1992,  p. 
131-134.  Research  supported  by  Clemson  Univ.. 

A  relaxation  labeling  algorithm  is  developed.  The 
major  advantage  of  this  algorithm  over  the  existing 
ones  is  that  the  mathematic  operation  is  simplified. 
The  simplification  eases  the  analysis  of  the  conver¬ 
gence  properties.  Both  the  theoretical  and  applica¬ 
tion  aspects  of  the  proposed  algorithm  are  investi¬ 
gated.  The  local  convergence  properties  of  a  label¬ 
ing  process  with  n  labels  and  m  labels  are  estab¬ 
lished.  The  investigation  of  the  interaction  among 
the  nodes  in  a  multinode  labeling  process  reveals 
some  insight  into  the  mathematical  issues  involved 
in  the  relaxation  operations. 

Intelligent  sensing  and  control  for  advanced 
teleoperation 

93A54158 

LEE,  SUKHAN  (JPL,  Pasadena;  Southern  Califor¬ 
nia  Univ.,  Los  Angeles,  CA)  IEEE  Control  Systems 
Magazine  (ISSN  0272-1708),  vol.  13,  no.  3,  June 
1993,  p.  19-28. 

A  theoretical  framework  is  presented  for  a 
'sensing-knowledge  command-fusion1  paradigm  of 
interactive  and  cooperative  sensing  and  control  in 
advanced  teleoperators,  which  takes  advantage  of 
both  current  and  projected  robotic  dexterousness 
and  sensor-based  autonomy  capabilities.  Attention  is 
given  to  (1)  a  method  for  the  achievement  of  a 
sensing-knowledge-command  computational  mecha¬ 
nism  that  implements  the  intended  cooperative/ 
interactive  system,  and  (2)  the  system  architecture 
and  man/machine-interface  protocols  entailed  by 
this  implementation. 

Experimental  object-level  strategic  control  with 
cooperating  manipulators 

93A53196 


SCHNEIDER,  STANLEY  A.;  CANNON,  ROBERT 
H.,  JR.  (Stanford  Univ.,  CA)  International  Journal 
of  Robotics  Research  (ISSN  0278-3649),  vol.  12, 
no.  4,  Aug.  1993,  p.  338-350. 

This  article  presents  the  high-level  control  module 
and  user  interface  of  the  Dynamic  and  Strategic 
Control  of  Cooperating  Manipulators  (DASCCOM) 
project  at  Stanford  University's  Aerospace  Robotics 
Laboratory.  In  addition  to  cooperative  dynamic 
control,  DASCCOM  incorporates  real-time  vision- 
ased  feedback,  a  novel  strategic  programming  tech¬ 
nique,  and  an  iconic  'object-only'  graphical  user 
interface.  By  focusing  on  the  vertical  integration 
problem,  we  are  examining  not  only  these  subsys¬ 
tems,  but  also  their  interfaces  and  interactions.  The 
control  system  implements  a  multilevel  hierarchical 
structure  interconnected  via  a  real-time  network.  At 
the  highest  level,  a  mouse-driven  graphical  user 
interface  allows  an  operator  to  direct  the  activities 
of  the  system  conceptually.  Strategic  command  is 
provided  by  an  event-driven  finite  state  machine. 
This  methodology  provides  a  powerful  yet  flexible 
technique  for  managing  concurrent  system  interac¬ 
tions.  The  dynamic  controller  implements  object 
impedance  control  -  an  extension  of  Neville  Hogan's 
impedance  control  concept  to  cooperative  multiple- 
arm  manipulation  of  a  single  object.  This  article 
concentrates  on  user  interfacing  techniques  and 
strategic  programming  capabilities.  These  modules 
allow  the  user  to  directly  specify  conceptual  object 
relationships.  Experimental  results  are  presented, 
showing  the  system  locating  and  identifying  a  mov¬ 
ing  object,  'catching'  it,  and  performing  a  simple 
cooperative  assembly.  Each  of  these  operations  is 
executed  autonomously,  with  only  object-level 
task-specification  direction  from  a  remote  operator. 

Mobile  robots  VI;  Proceedings  of  the  Meeting, 
Boston,  MA,  Nov.  14,  15,  1991 

93A53170 

WOLFE,  WILLIAM  J.;  CHUN,  WENDELL  H. 
(Colorado  Univ.,  Denver)  ;  (Martin  Marietta  Corp., 
Denver,  CO)  Bellingham,  WA,  Society  of 
Photo-Optical  Instrumentation  Engineers  (SPIE 
Proceedings.  Vol.  1613),  1992,  378  p.  (For  individ¬ 
ual  items  see  A93-53171  to  A93-53173) 

Various  papers  on  mobile  robots  are  presented. 
Individual  topics  addressed  include:  high-level 
modes  for  controlling  mobile  robots,  controlling  a 
mobile  robot  using  partial  representations,  batlike 
mobile  robot  for  tracking  a  moving  obstacle, 
self-awareness  in  mobile  robots,  autonomous  vehicle 
maneuvering,  survival  of  falling  robots,  junction 
detection  and  pathway  selection,  robot  path  planning 
based  on  variational  methods,  mobile  robot  localiza¬ 
tion  with  multiple  stationary  cameras,  robot  path 
planning  using  expert  systems  and  machine  vision, 
robot  mapping  in  unstructured  environments.  Also 
discussed  are:  model  for  shape  and  motion  percep¬ 
tion,  vision-based  method  for  autonomous  landing, 
multilayer  robust  estimation  for  motion  segmenta¬ 
tion,  robot  position  estimation  using  range  and  com¬ 
pass  data,  space-time  tradeoffs  for  adaptive 
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real-time  tracking,  new  vision-based  approach  to 
navigation,  vision  algorithm  for  mobile  vehicle 
navigation,  thermal  and  range  fusion  for  a  planetary 
rover. 


Joint-space  Lyapunov-based  direct  adaptive  control 
of  a  kinematically  redundant  telerobot  manipulator 

93A53038 

NGUYEN,  CHARLES  C.;  ZHOU,  ZHEN-LEI; 
MOSIER,  GARY  E.  (Catholic  Univ.  of  America, 
Washington);  (NASA,  Goddard  Space  Flight  Center, 
Greenbelt,  MD)  Control  and  Computers  (ISSN 
0315-8934),  vol.  21,  no.  1,  1993,  p.  23-27. 

This  paper  presents  the  design  of  a  joint-space 
adaptive  control  scheme  for  controlling  the  slave 
arm  motion  of  a  dual-arm  telerobot  system  devel¬ 
oped  at  Goddard  Space  Flight  Center  (GSFC)  to 
study  telerobotic  operations  in  space.  Each  slave 
arm  of  the  dual-arm  system  is  a  kinematically  re¬ 
dundant  manipulator  with  seven  degrees  of  freedom 
(DOF).  Using  the  concept  of  model  reference  adap¬ 
tive  control  (MRAC)  and  Liupunov  direct  method, 
we  derive  an  adaptation  algorithm  that  adjusts  the 
PD  controller  gains  of  the  control  scheme.  The 
development  of  the  adaptive  control  scheme  as¬ 
sumes  that  the  slave  arm  motion  is  non-compliant 
and  slowly  varying.  The  implementation  of  the 
derived  control  scheme  does  not  require  the  compu¬ 
tation  of  manipulator  dynamics  which  makes  the 
control  scheme  sufficiently  fast  for  real-time  appli¬ 
cations.  Computer  simulation  study  performed  for 
the  7-DOF  slave  arm  shows  that  the  developed 
control  scheme  can  efficiently  adapt  to  sudden 
change  in  payload  while  tracking  various  test  trajec¬ 
tories  such  as  ramp  or  sinusoids  with  negligible 
position  errors. 

Computer  vision  for  autonomous  robotics  in  space 

93A53031 

WONG,  A.  K.  C.  (Waterloo  Univ.,  Canada)  In: 
Visual  information  processing  II;  Proceedings  of  the 
Meeting,  Orlando,  FL,  Apr.  14-16,  1993 
(A93-53022  23-63).  Bellingham,  WA,  Society  of 
Photo-Optical  Instrumentation  Engineers,  1993,  p. 
176-191.  Research  supported  by  Manufacturing 
Research  Corp.  of  Ontario,  Thomson-CSF,  Spar 
Aerospace,  Ltd.,  et  al. 

A  robust  robot  vision  system  is  presented  and  its 
real-time  capability  for  carrying  out  vision  tasks  is 
presented.  The  system  can  synthesize  a  wire-frame 
model  of  a  3D  object  from  a  sequence  of  images  if 
the  camera  position  relative  to  each  of  the  images  is 
approximately  known.  Hence  such  a  task  can  be 
achieved  from  images  taken  by  a  camera  mounted 
on  a  robot  arm.  Once  the  model  is  obtained,  a 
rule-network  for  scene  interpretation  based  on  a 
hypothesis  refinement  procedure  can  be  constructed 
and  used  to  recognize,  locate,  and  track  the  object. 

A  variety  of  experiments  are  used  to  show  that  such 
a  capability  has  a  number  of  applications  in  the 
space  robot  project. 


Motion  planning  of  a  dual-aim  free-floating 
manipulator  with  inertially  fixed  base 

93A51450 

AGRAWAL,  SUNIL  K.;  SHIRUMALLA, 
SHRAVAN  (Ohio  Univ.,  Athens)  In:  AIAA  Guid¬ 
ance,  Navigation  and  Control  Conference, 

Monterey,  CA,  Aug.  9-11,  1993,  Technical  Papers. 
Pt.  3  (A93-51301  22-63).  Washington,  American 
Institute  of  Aeronautics  and  Astronautics,  1993,  p. 
1472-1481.  Research  supported  by  NSF. 

A  scheme  for  joint  and  Cartesian  motion  planning 
of  a  dual  arm  free-floating  planar  roots  is  presented 
using  position  and  rate  kinematic  equations  so  that 
the  base  of  the  robot  remains  inertially  fixed.  Even 
though  free-floating  manipulators  are  characterized 
by  nonholonomic  constraints,  it  is  shown  that  the 
inverse  position  kinematics  coupled  with  an  iterative 
search  procedure  results  in  identical  path  predicted 
by  direct  integration  of  the  rate  equations.  It  is 
shown  that  singularities  can  be  easily  avoided. 

Adaptive  spline  networks  for  estimating  camera 
control  parameters  in  robot-vision  system 

93A50760 

LAYNE,  J.  D.  (Martin  Marietta  Astronautics  Group, 
Denver,  CO)  In:  WNN  92;  Proceedings  of  the  3rd 
Workshop  on  Neural  Networks:  Acadern- 
ic/Industrial/NASDefense,  Auburn  Univ.,  AL,  Feb. 
10-12,  1992  and  South  Shore  Harbour,  TX,  Nov. 
4-6,  1992  (A93-50726  21-63).  San  Diego, 
CBellingham,  WA,  Society  for  Computer  Simula¬ 
tion/Society  of  Photo-Optical  Instrumentation  Engi¬ 
neers,  1993,  p.  349-354. 

Image  centroids  are  frequently  used  to  accurately 
locate  features  in  images  in  many  computer  vision 
problems.  However,  non-linear  effects  in  the  pro¬ 
jection  of  object  features  to  the  image  plane  can 
cause  significant  errors  in  the  centroids,  and  hence 
errors  in  pose  estimation  and  camera  calibration. 
This  paper  describes  a  neural  network  that  predicts 
centroid  errors  due  to  surface  tilt  and  lens  distortion. 
The  approach  is  based  on  multivariable  function 
approximation.  Backpropagation  networks  trained 
too  slowly  and  generalized  poorly  at  estimating  the 
camera  control  parameters,  hence  an  adaptive  spline 
network  (MARS)  was  implemented  that  meets  accu¬ 
racy  and  performance  requirements.  MARS  incorpo¬ 
rates  cubic  spline  basis  functions  with  recursive 
partitioning  of  the  input  space,  uses  statistical  learn¬ 
ing  (least  squares)  instead  of  gradient  descent,  and 
predicts  performance  on  test  data  during  training 
using  a  cross-validation  model  to  improve  values  of 
network  parameters. 

A  manipulator  control  testbed  -  Implementation  and 
applications 

93A50594 

FIALA,  JOHN;  WAVERING,  ALBERT;  LUMIA, 
RONALD  (NIST,  Gaithersburg,  MD)  In:  Guidance 
and  control  1992;  Proceedings  of  the  15th  Annual 
AAS  Rocky  Mountain  Conference,  Keystone,  CO, 
Feb.  8-12,  1992  (A93-50576  21-18).  San  Diego, 

CA,  Univelt,  Inc.,  1992,  p.  319-334. 
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An  implementation  of  the  lower  levels  of  the 
NASNIST  Standard  Reference  Model  (NASREM) 
Telerobot  Control  System  Architecture  has  been 
developed  at  NIST.  The  implementation  includes 
manipulator  servo  control,  rate  teleoperation,  auton¬ 
omous  trajectory  generation,  and  visual  sensing. 

This  paper  describes  how  the  system  is  designed  to 
be  a  testbed  for  manipulator  control  via  generic 
interfaces  and  a  modular  Ada  software  architecture. 
The  multiprocessor  hardware  architecture  which 
supports  the  software  architecture  for  real-time 
operation  is  also  described.  The  paper  presents 
applications  of  the  testbed  system  to  specific  manip¬ 
ulator  control  problems,  including  some  example 
comparisons  of  different  strategies  for  servo  control 
and  trajectory  generation. 

Telerobot  control  mode  performance  assessment 

93A50593 

ZIMMERMAN,  WAYNE;  BACKES,  PAUL; 
CHIRIKJIAN,  GREG  (JPL,  Pasadena,  CA);  (Cali¬ 
fornia  Inst,  of  Technology,  Pasadena)  In:  Guidance 
and  control  1992;  Proceedings  of  the  15th  Annual 
AAS  Rocky  Mountain  Conference,  Keystone,  CO, 
Feb.  8-12,  1992  (A93-50576  21-18).  San  Diego, 

CA,  Univelt,  Inc.,  1992,  p.  305-318. 

With  the  maturation  of  various  developing  robot 
control  schemes,  it  is  becoming  extremely  important 
that  the  technical  community  evaluate  the  perfor¬ 
mance  of  these  various  control  technologies  against 
an  established  baseline  to  determine  which  technol¬ 
ogy  provides  the  most  reliable  robust,  and  safe  on- 
orbit  robot  control.  The  Supervisory  Telerobotics 
Laboratory  (STELER)  at  JPL  has  developed  a 
unique  robot  control  capability  which  has  been 
evaluated  by  the  NASA  technical  community  and 
found  useful  for  augmenting  both  the  operator  inter¬ 
face  and  control  of  intended  robotic  systems  on¬ 
board  the  Space  Station.  As  part  of  the  technology 
development  and  prototyping  effort,  the  STELER 
team  has  been  evaluating  the  performance  of  differ¬ 
ent  control  modes;  namely,  teleoperation  under 
position,  or  rate,  control,  teleoperation  with  force 
reflection  and  shared  control.  Nine  trained  subjects 
were  employed  in  the  performance  evaluation  in¬ 
volving  several  high  fidelity  servicing  tasks.  Four 
types  of  operator  performance  data  were  collected; 
task  completion  time,  average  force,  peak  force, 
and  number  of  operator  successes  and  errors.  This 
paper  summarizes  the  results  of  this  performance 
evaluation. 

Ground-remote  control  for  space  station  telerobotics 
with  time  delay 

93A50592 

BACKES,  PAUL  G.  (JPL,  Pasadena,  CA)  In:  Guid¬ 
ance  and  control  1992;  Proceedings  of  the  15th 
Annual  AAS  Rocky  Mountain  Conference,  Key¬ 
stone,  CO,  Feb.  8-12,  1992  (A93-50576  21-18).  San 
Diego,  CA,  Univelt,  Inc.,  1992,  p.  285-303. 

The  study  proposes  a  ground-remote  telerobot  con¬ 
trol  architecture  which  could  be  used  for  control  of 
Space  Station  Freedom  manipulators.  The  architec¬ 


ture  provides  two  local-site  operator  control  stations 
representing  potential  earth-based  and  remote  Space 
Station-based  operator  control  stations.  A  unified 
control  system  at  the  remote  site  provides  autono¬ 
mous,  shared,  and  teleoperation  control  for 
single-and  dual-arm  task  execution.  An  operational 
laboratory  system  which  demonstrates  the  feasibility 
of  various  technologies  in  the  proposed  architecture, 
including  teleoperation,  shared  control,  and  super¬ 
vised  autonomy,  is  described.  Enhancements  to  the 
system  currently  under  development,  including  re¬ 
mote  site  implementation  in  Ada,  integration  and 
control  of  a  redundant  7-DOF  manipulator,  and 
local  site  advanced  operator  aids,  are  also  described. 

Interactive  and  cooperative  sensing  and  control  for 
advanced  teleoperation 

93A49443 

LEE,  SUKHAN;  ZAPATA,  EDUARDO; 
SCHENKER,  PAUL  S.  (JPL,  Pasadena;  Southern 
California  Univ.,  Los  Angeles,  CA);  (Southern 
California  Univ.,  Los  Angeles,  CA);  (JPL,  Pasade¬ 
na,  CA)  In:  Sensor  fusion  IV:  Control  paradigms 
and  data  structures;  Proceedings  of  the  Meeting, 
Boston,  MA,  Nov.  12-15,  1991  (A93-49438  21-63). 
Bellingham,  WA,  Society  of  Photo-Optical  Instru¬ 
mentation  Engineers,  1992,  p.  516-530. 

This  paper  presents  the  paradigm  of  interactive  and 
cooperative  sensing  and  control  as  a  fundamental 
mechanism  of  integrating  and  fusing  the  strengths 
of  man  and  machine  for  advanced  teleoperation. 

The  interactive  and  cooperative  sensing  and  control 
is  considered  as  an  extended  and  generalized  form 
of  traded  and  shared  control.  The  emphasis  of  inter¬ 
active  and  cooperative  sensing  and  control  is  given 
to  the  distribution  of  mutually  nonexclusive 
subtasks  to  man  and  machine,  the  interactive  invo¬ 
cation  of  subtasks  under  the  man/machine  symbiotic 
relationship,  and  the  fusion  of  information  and 
decision-making  between  man  and  machine  accord¬ 
ing  to  their  confidence  measures.  The  proposed 
interactive  and  cooperative  sensing  and  control 
system  is  composed  of  such  major  functional  blocks 
as  the  logical  sensor  system,  the  sensor-based  local 
autonomy,  the  virtual  environment  formation,  and 
the  cooperative  decision-making  between  man  and 
machine.  A  case  study  is  performed  to  demonstrate 
the  feasibility  of  implementing  the  fundamental 
theory  and  system  architecture  of  interactive  and 
cooperative  sensing  and  control,  proposed  for  the 
new  generation  of  teleoperation. 

Continuous  motion  using  task-directed  stereo  vision 

93A49442 

GAT,  ERANN;  LOCH,  JOHN  L.  (JPL,  Pasadena, 
CA)  In:  Sensor  fusion  IV:  Control  paradigms  and 
data  structures;  Proceedings  of  the  Meeting,  Boston, 
MA,  Nov.  12-15,  1991  (A93-49438  21-63). 
Bellingham,  WA,  Society  of  Photo-Optical  Instru¬ 
mentation  Engineers,  1992,  p.  294-298. 

The  performance  of  autonomous  mobile  robots 
performing  complex  navigation  tasks  can  be  dra¬ 
matically  improved  by  directing  expensive  sensing 
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and  planning  in  service  of  the  task.  The 
task-direction  algorithms  can  be  quite  simple.  In  this 
paper  we  describe  a  simple  task-directed  vision 
system  which  has  been  implemented  on  a  real  out¬ 
door  robot  which  navigates  using  stereo  vision. 
While  the  performance  of  this  particular  robot  was 
improved  by  task-directed  vision,  the  performance 
of  task-directed  vision  in  general  is  influenced  in 
complex  ways  by  many  factors.  We  briefly  discuss 
some  of  these,  and  present  some  initial  simulated 
results. 

Operator  performance  with  alternative  manual 
control  modes  in  teleoperation 

93A49397 

DAS,  H.;  ZAK,  H.;  KIM,  W.  S.;  BEJCZY,  A.  K.; 
SCHENKER,  P.  S.  (JPL,  Pasadena,  CA)  Presence: 
Teleoperators  and  Virtual  Environments  (ISSN 
1054-7460),  vol.  1,  no.  2,  Spring  1992,  p.  201-218. 
Recent  experiments  conducted  at  the  JPL  comparing 
alternative  manual  control  modes  using  the  JPL 
Advanced  Teleoperator  system  are  described.  Of 
particular  interest  were  control  modes  that  provide 
force  reflection  to  the  operator.  The  task  selected 
for  the  experiment  is  a  portion  of  the  Solar  Maxi¬ 
mum  Satellite  Repair  procedure  we  developed  to 
demonstrate  the  repair  of  the  Solar  Maximum  Sat¬ 
ellite  with  teleoperators.  The  seven  manual  control 
modes  evaluated  in  the  experiment  are  combinations 
of  manual  position  or  resolved  motion  rate  control 
with  alternative  control  schemes  for  force  reflection 
and  remote  manipulator  compliance.  Performance 
measures  used  were  task  completion  times,  average 
force  and  torque  exerted  during  the  execution  of  the 
task,  and  cumulative  force  and  torque  exerted.  The 
results  were  statistically  analyzed  and  they  show 
that,  in  general,  force  reflection  significantly  im¬ 
proves  operator  performance  and  indicate  that  a 
specific  force-reflecting  scheme  may  yield  the  best 
performance  among  the  control  modes  we  tested. 
Also,  our  experiment  showed  that,  for  the  selected 
task,  the  position  control  modes  were  preferable  to 
the  rate  control  modes  and  slave  manipulator  com¬ 
pliance  reduced  task  interaction  forces  and  torques. 

Transforming  human  hand  motion  for 
telemanipulation 

93A49394 

SPEETER,  THOMAS  H.  (AT&T  Bell  Labs., 
Holmdel,  NJ)  Presence:  Teleoperators  and  Virtual 
Environments  (ISSN  1054-7460),  vol.  1,  no.  1, 
Winter  1992,  p.  63-79. 

Manipulation  by  teleoperation  (telemanipulation) 
offers  an  apparently  straightforward  and  less 
computationally  expensive  route  toward  dextrous 
robotic  manipulation  than  automated  control  of 
multi  fingered  robotic  hands.  The  functional  trans¬ 
formation  of  human  hand  motions  into  equivalent 
robotic  hand  motions,  however,  presents  both  con¬ 
ceptual  and  analytical  problems.  This  paper  reviews 
and  proposes  algorithmic  methods  for  transforming 
the  actions  of  human  hands  into  equivalent  actions 
of  slave  multifingered  robotic  hands.  Forward  posi¬ 


tional  transformation  is  considered  only,  the  design 
of  master  devices,  feedforward  dynamics,  and  force 
feedback  are  not  considered  although  their  impor¬ 
tance  for  successful  telemanipulation  is  understood. 
Linear,  nonlinear,  and  functional  mappings  are  dis¬ 
cussed  along  with  performance  and  computational 
considerations. 

Teleprogramming  -  Toward  delay-invariant  remote 
manipulation 

93A49392 

FUNDA,  JANEZ;  LINDSAY,  THOMAS  S.;  PAUL, 
RICHARD  P.  (Pennsylvania  Univ.,  Philadelphia) 
Presence:  Teleoperators  and  Virtual  Environments 
(ISSN  1054-7460),  vol.  1,  no.  1,  Winter  1992, 
p.  29-44. 

The  paper  proposes  a  control  methodology,  called 
teleprogramming,  which  is  a  new  approach  to  low- 
level  task  specification  for  a  remotely  located 
robotic  system  and  which  offers  a  practical  interme¬ 
diate  solution  to  time-delayed  remote  manipulation. 
Teleprogramming  combines  the  power  of  a  graphic 
previewing  display  with  the  provision  of  real-time 
kinesthetic  feedback,  to  allow  the  operator  to  inter¬ 
actively  define  the  task  to  be  performed  remotely. 
The  instructions  are  generated  automatically,  in  real 
time,  based  on  the  operator's  interaction  with  the 
simulated  environment.  The  slave  robot  executes 
these  commands  delayed  in  time,  making  it  possible 
for  the  operator  to  correct  actions  in  case  of  an 
error.  The  paper  describes  the  main  components 
of  the  teleprogramming  system  and  reports  the 
preliminary  results  from  the  use  of  an  experimental 
system. 

Intelligent  robotics;  Proceedings  of  the  International 
Symposium,  Bangalore,  India,  Jan.  2-5,  1991 

93A49350 

VIDYASAGAR,  M.;  TRIVEDI,  MOHAN  M. 

(Centre  for  Artificial  Intelligence  and  Robotics, 
Bangalore,  India);  (Tennessee  Univ.,  Knoxville) 

New  Delhi,  Tata  McGraw-Hill  Publishing  Co.,  Ltd. 
(SPIE  Proceedings.  Vol.  1571),  1991,  746  p.  (For 
individual  items  see  A93-49351  to  A93-49360) 

The  present  volume  on  intelligent  robotics  discusses 
parallel  vision  algorithms  using  sparse  array  repre¬ 
sentations,  vision-based  techniques  for  rotorcraft 
low-altitude  flight,  the  design  of  direct-drive  robots 
using  indigenously  developed  dc  torque  motors,  and 
an  end-effector  for  3D  manipulation  of  multiple-ply 
apparel  workpieces.  Attention  is  given  to  a  robot 
vision  algorithm  for  manipulating  objects  in  a  clut¬ 
tered  scene,  a  sensor-based  part  feeding  gate,  the 
development  of  a  four-axis  robot  for  automation  in 
the  nuclear  industry,  and  the  design  of  a  composite 
controller  for  a  two-link  flexible  manipulator.  Top¬ 
ics  addressed  include  two-time  scale  control  for  an 
arm  with  joint  and  link  compliance,  point-to-point 
control  of  elastic  joint  robots,  intelligent  robotic 
polishing,  and  a  robotic  system  for  the  inspection  of 
turbine  disks.  Also  discussed  are  measures  of  inten¬ 
sity  of  collision  between  convex  objects  and  their 
efficient  computation,  efficient  coordinated  motion, 
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observer-based  control  laws  for  robotic  manipula¬ 
tors,  and  multipolynomial  resultant  algorithms. 

Controlled  motion  in  an  elastic  world 

93A46742 

BOOK,  WAYNE  J.  (Georgia  Inst,  of  Technology, 
Atlanta)  ASME,  Transactions,  Journal  of  Dynamic 
Systems,  Measurement,  and  Control  (ISSN 
0022-0434),  vol.  115,  no.  2(B),  June  1993,  p. 
252-261.  (Previously  announced  in  STAR  as 
N93- 18778) 

The  flexibility  of  the  drives  and  structures  of  con¬ 
trolled  motion  systems  are  presented  as  an  obstacle 
to  be  overcome  in  the  design  of  high  performance 
motion  systems,  particularly  manipulator  arms.  The 
task  and  the  measure  of  performance  to  be  applied 
determine  the  technology  appropriate  to  overcome 
this  obstacle.  Included  in  the  technologies  proposed 
are  control  algorithms  (feedback  and  feed  forward), 
passive  damping  enhancement,  operational  strate¬ 
gies,  and  structural  design.  Modeling  of  the  distrib¬ 
uted,  nonlinear  system  is  difficult,  and  alternative 
approaches  are  discussed.  The  author  presents  per¬ 
sonal  perspectives  on  the  history,  status,  and  future 
directions  in  this  area. 

Kinematics  and  control  of  a  fully  parallel 
force-reflecting  hand  controller  for  manipulator 
teleoperation 

93A45598 

BRYFOGLE,  MARK  D.;  NGUYEN,  CHARLES 
C.;  ANTRAZI,  SAMI  S.;  CHIOU,  PETER  C.  (Sci¬ 
ence  Applications  International  Corp.,  McLean, 

VA);  (Catholic  Univ.  of  America,  Washington) 
Journal  of  Robotic  Systems  (ISSN  0741-2223),  vol. 
10,  no.  5,  July  1993,  p.  745-766. 

Design  of  a  parallel  force-reflecting  hand  controller 
that  implements  a  friction-  and  inertia  canceling 
control  loop  about  the  entire  mechanism  based  on 
wrench  sensing  in  the  mechanism  handgrip  is  dis¬ 
cussed.  Kinematics  of  the  controller  under  consid¬ 
eration  is  analyzed  and  results  are  presented  using  a 
closed-form  solution  for  the  inverse  kinematics  and 
Newton-Raphson's  method  for  the  forward  kinemat¬ 
ics.  Results  indicate  that  the  force  control  scheme 
based  on  a  handgrip  force  sensor  provides  smaller 
steady-state  errors  than  the  scheme  without  a 
handigrip  sensor. 

Adaptive  control  of  a  Stewart  platform-based 
manipulator 

93A45596 

NGUYEN,  CHARLES  C.;  ANTRAZI,  SAMI  S.; 
ZHOU,  ZHEN-LEI;  CAMPBELL,  CHARLES  E, 

JR.  (Catholic  Univ.  of  America,  Washington); 
(NASA,  Goddard  Space  Flight  Center,  Greenbelt, 
MD)  Journal  of  Robotic  Systems  (ISSN  0741-2223), 
vol.  10,  no.  5,  July  1993,  p.  657-687. 

A  joint-space  adaptive  control  scheme  for  control¬ 
ling  noncompliant  motion  of  a  Stewart 
platform-based  manipulator  (SPBM)  was  imple¬ 
mented  in  the  Hardware  Real-Time  Emulator  at 
Goddard  Space  Flight  Center.  The  six-degrees  of 


freedom  SPBM  uses  two  platforms  and  six  linear 
actuators  driven  by  dc  motors.  The  adaptive  control 
scheme  is  based  on  proportional-derivative  control¬ 
lers  whose  gains  are  adjusted  by  an  adaptation  law 
based  on  model  reference  adaptive  control  and 
Liapunov  direct  method.  It  is  concluded  that  the 
adaptive  control  scheme  provides  superior  tracking 
capability  as  compared  to  fixed-gain  controllers. 

Attitude  control  algorithm  for  free-flying  space 
robot  using  thruster 

93A45274 

KOBAYASHI,  NOBUYUKI;  SAITO,  OSAMU; 
YOSHIE,  YUUKI;  KURIHARA,  HIROSHI 
Ishikawajima-Harima  Engineering  Review  (ISSN 
0578-7904),  vol.  33,  no.  1,  Jan.  1993,  p.  1-6. 

An  attitude  control  algorithm  for  a  free-flying  space 
robot  to  control  thrusters  is  proposed.  The  motion  of 
the  manipulator  on  a  space  robot  causes  attitude 
deviation  of  the  main  body  because  of  dynamic 
interaction.  The  proposed  Disturbed-Torque  Com¬ 
pensation  Algorithm,  a  kind  of  feedforward  control, 
considers  the  inertia  force,  calculated  by  the  equa¬ 
tion  of  motion,  as  the  disturbance  torque  for  the 
main  body.  The  algorithm  activates  thrusters  to 
compensate  for  the  torque.  The  proposed  algorithm 
is  compared  to  the  Time  or  Time-Energy  Optimal 
Control  algorithm.  Simulation  results  for  attitude 
deviations  and  input  energy  are  discussed. 

Linearization  of  manipulator  dynamics  using  spatial 
operators 

93A43725 

JAIN,  A.;  RODRIGUEZ,  G.  (JPL,  Pasadena,  CA) 
IEEE  Transactions  on  Systems,  Man,  and  Cyber¬ 
netics  (ISSN  0018-9472),  vol.  23,  no.  1,  Jan.-Feb. 
1993,  p.  239-248. 

Linearized  dynamics  models  for  manipulators  are 
useful  in  robot  analysis,  motion  planning,  and  con¬ 
trol  applications.  Techniques  from  the  spatial  oper¬ 
ator  algebra  are  used  to  obtain  closed  form  operator 
expressions  for  two  types  of  linearized  dynamics 
models,  the  linearized  inverse  and  forward  dynamics 
models.  Spatially  recursive  algorithms  of  O(n)  and 
O(n-squared)  complexity  for  the  computation  of  the 
perturbation  vector  and  coefficient  matrices  for  the 
linearized  inverse  dynamics  model  are  developed 
first.  Subsequently,  operator  factorization  and  inver¬ 
sion  identities  are  used  to  develop  corresponding 
closed-form  expressions  for  the  linearized  forward 
dynamics  model  (LFDM).  Once  again,  these  are 
used  to  develop  algorithms  of  O(n)  and 
O(n-squared)  complexity  for  the  computation  of  the 
perturbation  vector  and  the  coefficient  matrices.  The 
algorithms  for  the  LFDM  do  not  require  the  explicit 
computation  of  the  mass  matrix  nor  its  numerical 
inversion  and  are  also  of  lower  complexity  than  the 
conventional  O(n-cubed)  algorithms. 

Real-time  collision  avoidance  in  teleoperated 
whole-sensitive  robot  arm  manipulators 

93A43723 

LUMELSKY,  VLADIMIR  J.;  CHEUNG, 
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EDWARD  (Wisconsin  Univ.,  Madison);  (NASA, 
Goddard  Space  Flight  Center,  Greenbelt,  MD)  IEEE 
Transactions  on  Systems,  Man,  and  Cybernetics 
(ISSN  0018-9472),  vol.  23,  no.  1,  Jan.-Feb.  1993,  p. 
194-203. 

A  hybrid  robot  teleoperation  system  is  presented 
which  makes  use  of  the  methodology  of  motion 
planning  for  whole-sensitive  robots  to  assist  the 
operator  in  generating  collision-free  motion  in  a 
master-slave  robot  arm  manipulator  system.  The 
system  combines  operator  commands  with  data 
from  the  sensitive  skin  to  guarantee  safe  motion  for 
the  entire  body  of  the  robot  arm.  The  arm  avoids 
obstacles  automatically  and  in  real  time  and  moves 
in  a  collision-free  manner  although  no  prior  knowl¬ 
edge  of  the  objects  in  the  environment  is  available 
to  the  motion  planning  system  and  no  constraints 
are  imposed  on  the  obstacle  shapes.  The  operator  is 
thus  relieved  of  the  task  of  providing  safety  of  the 
robot  arm  and  surrounding  objects. 

Visual  specification  of  robot  motion 

93A42845 

SHIU,  Y.  C.;  CHONG,  R.;  RUNNER,  K.; 

SCAGGS,  T.;  SETH,  N.;  CRAVEN,  R.  (Wright 
State  Univ.,  Dayton,  OH);  (USAF,  Aeronautical 
Systems  Div.,  Wright-Patterson  AFB,  OH)  In: 
NAECON  92;  Proceedings  of  the  IEEE  1992  Na¬ 
tional  Aerospace  and  Electronics  Conference,  Day- 
ton,  OH,  May  18-22,  1992.  Vol.  2  (A93-42776 
17-01).  New  York,  Institute  of  Electrical  and  Elec¬ 
tronics  Engineers,  Inc.,  1992,  p.  705-708.  Research 
supported  by  NASA  and  Ohio  Aerospace  Inst. 

The  authors  describe  the  use  of  stereo  pairs  of  im¬ 
ages  to  specify  robot  motion.  The  experimental 
setup  includes  a  SUN  workstation,  a  PUMA  560 
robot,  and  an  Imaging  151  vision  system.  An 
X-window  environment  displays  stereo  images  of 
the  work  scene.  Image  processing  is  performed  to 
extract  linear  edge  segments  from  the  images  and 
the  results  are  displayed  on  screen.  Using  a  pointing 
device,  the  user  selects  a  group  of  edges  from  the 
object  relevant  to  the  task.  The  3D  structure  of  this 
group  of  features  is  found  by  stereo  triangulation 
and  they  can  be  displayed  in  3D  from  any  point  of 
view.  A  viewpoint  orthogonal  to  the  plane  defined 
by  these  3D  edges  is  used  to  specify  the  robot  posi¬ 
tion  relative  to  object  position.  The  actual  robot  will 
then  be  moved  to  the  specified  position. 

Principles  of  control  for  robotic  excavation 

93A42097 

BERNOLD,  LEONHARD  E.  (North  Carolina  State 
Univ.,  Raleigh)  In:  Engineering,  construction,  and 
operations  in  space  III:  Space  '92;  Proceedings  of 
the  3rd  International  Conference,  Denver,  CO,  May 
31 -June  4,  1992.  Vol.  2  (A93-41976  17-12).  New 
York,  American  Society  of  Civil  Engineers,  1992, 
p.  1401-1412. 

The  issues  of  automatic  planning  and  control  sys¬ 
tems  for  robotic  excavation  are  addressed.  Attention 
is  given  to  an  approach  to  understanding  the  princi¬ 
ples  of  path  and  motion  control  which  is  based  on 


scaled  modeling  and  experimentation  with  different 
soil  types  and  soil  conditions.  Control  concepts  for 
the  independent  control  of  a  bucket  are  discussed, 
and  ways  in  which  force  sensors  could  provide  the 
necessary  data  are  demonstrated.  Results  of  experi¬ 
ments  with  lunar  simulant  showed  that  explosive 
loosening  has  a  substantial  impact  on  the  energy 
needed  during  excavation.  It  is  argued  that  through 
further  laboratory  and  field  research,  'pattern 
languages'  for  different  excavators  and  soil  condi¬ 
tions  could  be  established  and  employed  for  robotic 
excavation. 

Contact  sensing  from  force  measurements 

93A41673 

BICCHI,  ANTONIO;  SALISBURY,  J.  K.;  BROCK, 
DAVID  L.  (MIT,  Cambridge,  MA)  International 
Journal  of  Robotics  Research  (ISSN  0278-3649), 
vol.  12,  no.  3,  June  1993,  p.  249-262.  Research 
supported  by  Systems  Development  Foundation, 
Sandia  National  Labs.,  DARPA,  CNR,  and  NATO. 
This  article  addresses  contact  sensing  (i.e.,  the  prob¬ 
lem  of  resolving  the  location  of  a  contact,  the  force 
at  the  interface,  and  the  moment  about  the  contact 
normals).  Called  'intrinsic'  contact  sensing  for  the 
use  of  internal  force  and  torque  measurements,  this 
method  allows  for  practical  devices  that  provide 
simple,  relevant  contact  information  in  practical 
robotic  applications.  Such  sensors  have  been  used  in 
conjunction  with  robot  hands  to  identify  objects, 
determine  surface  friction,  detect  slip,  augment 
grasp  stability,  measure  object  mass,  probe  surfaces, 
and  control  collision  and  for  a  variety  of  other  use¬ 
ful  tasks.  This  article  describes  the  theoretical  basis 
for  their  operation  and  provides  a  framework  for 
future  device  design. 

Fuzzy  logic  for  adaptive  control  of  complex  robots 
and  tele  robots 

93A40664 

FRANKE,  ERNEST  A.;  NEDUNGADI,  ASHOK 
(Southwest  Research  Inst.,  San  Antonio,  TX)  Soci¬ 
ety  of  Manufacturing  Engineers,  Maintaining  and 
Supporting  an  Aircraft  Fleet  Conference,  Dayton, 
OH,  June  9-11,  1992.  8  p.  Research  supported  by 
Southwest  Research  Inst. 

A  recently  completed  research  program  at  South¬ 
west  Research  Institute  has  developed  a  fuzzy  logic 
controller  for  a  redundant,  four  degree-of-freedom, 
planar  manipulator.  The  manipulator  end  point  tra¬ 
jectory  can  be  specified  by  either  a  computer  pro¬ 
gram  (robot  mode)  or  by  manual  input  (teleoperator 
mode).  The  approach  used  expresses  end-point  error 
the  location  of  manipulator  joints  and  proximity  to 
obstacles  as  fuzzy  variables.  Joint  motions  are  de¬ 
termined  by  a  fuzzy  rule  set  without  requiring  solu¬ 
tion  of  the  inverse  kinematic  equations.  Additional 
rules  for  sensor  data  and  preferred  manipulator 
configuration,  eg.  'righty'  or  'lefty',  are  easily  ac¬ 
commodated.  The  procedure  used  to  generate  the 
fuzzy  rules  can  be  extended  to  higher  DOF  systems. 
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Robotic  aircraft  painting  with  SAFARI 

93A40662 

BERRY,  HENRY  K.;  MATTHEWS,  JOHN  D. 
(Engineering,  Inc.,  Hampton,  VA)  Society  of  Man¬ 
ufacturing  Engineers,  Maintaining  and  Supporting 
an  Aircraft  Fleet  Conference,  Dayton,  OH,  June 
9-11,  1992.  11  p. 

This  paper  discusses  concept  through  installation 
and  operating  results  of  robotic  aircraft  painting  at 
Robins  AFB  using  a  SAFARI  to  wash,  prep,  and 
paint  F-15  aircraft  with  polyurethane  and  water¬ 
borne  high-solid  paints.  The  paper  includes  a  dis¬ 
cussion  of  how  the  same  robotic  system  can  be  used 
for  stripping  aircraft. 

Operator  assistant  systems  -  An  experimental 
approach  using  a  telerobotics  application 

93A39400 

BOY,  GUY  A.;  MATHE,  NATHALIE  (European 
Inst,  of  Cognitive  Sciences  and  Engineering, 

Labage,  France);  (NASA,  Ames  Research  Center, 
Moffett  Field,  CA)  International  Journal  of  Intelli¬ 
gent  Systems  (ISSN  0884-8173),  vol.  8,  1993,  p. 
271-286. 

This  article  presents  a  knowledge-based  system 
methodology  for  developing  operator  assistant  (OA) 
systems  in  dynamic  and  interactive  environments. 
This  is  a  problem  both  of  training  and  design, 
which  is  the  subject  of  this  article.  Design  includes 
both  design  of  the  system  to  be  controlled  and  de¬ 
sign  of  procedures  for  operating  this  system.  A 
specific  knowledge  representation  is  proposed  for 
representing  the  corresponding  system  and  opera¬ 
tional  knowledge.  This  representation  is  based  on 
the  situation  recognition  and  analytical  reasoning 
paradigm.  It  tries  to  make  explicit  common  factors 
involved  in  both  human  and  machine  intelligence, 
including  perception  and  reasoning.  An  OA  system 
based  on  this  representation  has  been  developed  for 
space  telerobotics.  Simulations  have  been  carried 
out  with  astronauts  and  the  resulting  protocols  have 
been  analyzed.  Results  show  the  relevance  of  the 
approach  and  have  been  used  for  improving  the 
knowledge  representation  and  the  OA  architecture. 

Adaptive  robotic  visual  tracking  -  Theory  and 
experiments 

93  A3  8206 

PAPANIKOLOPOULOS,  NIKOLAOS  P.; 

KHOSLA,  PRADEEP  K.  (Minnesota  Univ.,  Minne¬ 
apolis);  (Carnegie  Mellon  Univ.,  Pittsburgh,  PA) 
IEEE  Transactions  on  Automatic  Control  (ISSN 
0018-9286),  vol.  38,  no.  3,  March  1993,  p.  429-445. 
This  paper  addresses  the  use  of  a  vision  sensor  in 
the  feedback  loop  within  the  Controlled  Active 
Vision  framework.  Using  this  framework,  algo¬ 
rithms  are  proposed  for  the  solution  of  the  robotic 
(eye-in-hand  configuration)  visual  tracking  and 
servoing  problem.  We  state  the  problem  of  visual 
tracking  as  a  problem  of  combining  control  with 
computer  vision.  We  use  the  sum-of-squared  differ¬ 
ences  (SSD)  optical  flow  for  the  computation  of  the 
vector  of  discrete  displacements.  The  measurements 


can  be  derived  either  from  a  single  big  window  or 
from  multiple  small  windows.  These  displacements 
are  fed  to  an  adaptive  controller  (self-tuning  regula¬ 
tor)  that  creates  commands  for  a  robot  control  sys¬ 
tem.  The  whole  algorithm  is  based  on  the  on-line 
estimation  of  the  relative  distance  of  the  target  with 
respect  to  the  camera.  An  important  contribution  of 
this  work  is  that  it  requires  only  partial  knowledge 
of  the  relative  distance  of  the  target  with  respect  to 
the  camera.  This  fact  obviates  the  need  for  off-line 
calibration  of  the  eye-in-hand  robotic  system.  We 
have  implemented,  both  in  simulation  and  in  experi¬ 
ments,  three  different  adaptive  control  schemes,  and 
the  results  are  presented  in  this  paper.  The  computa¬ 
tional  complexity  and  the  experimental  results  dem¬ 
onstrate  that  the  proposed  algorithms  can  be  imple¬ 
mented  in  real  time. 

A  new  neural  net  approach  to  robot  3D  perception 
and  visuo-motor  coordination 

93A37003 

LEE,  SUKHAN  (JPL,  Pasadena;  Southern  Califor¬ 
nia  Univ.,  Los  Angeles,  CA)  In:  IJCNN  -  Interna¬ 
tional  Joint  Conference  on  Neural  Networks,  Balti¬ 
more,  MD,  June  7-11,  1992,  Proceedings.  Vol.  1 
(A93-37001  14-63).  New  York,  Institute  of  Electri¬ 
cal  and  Electronics  Engineers,  Inc.,  1992,  p.  1-299 
to  1-307. 

A  novel  neural  network  approach  to  robot  hand-eye 
coordination  is  presented.  The  approach  provides  a 
true  sense  of  visual  error  servoing,  redundant  arm 
configuration  control  for  collision  avoidance,  and 
invariant  visuo-motor  learning  under  gazing  control. 
A  3-D  perception  network  is  introduced  to  represent 
the  robot  internal  3-D  metric  space  in  which  visual 
error  servoing  and  arm  configuration  control  are 
performed.  The  arm  kinematic  network  performs 
the  bidirectional  association  between  3-D  space  arm 
configurations  and  joint  angles,  and  enforces  the 
legitimate  arm  configurations.  The  arm  kinematic 
net  is  structured  by  a  radial-based  competitive  and 
cooperative  network  with  hierarchical  self-organiz¬ 
ing  learning.  The  main  goal  of  the  present  work  is 
to  demonstrate  that  the  neural  net  representation  of 
the  robot  3-D  perception  net  serves  as  an  important 
intermediate  functional  block  connecting  robot  eyes 
and  arms. 

Application  of  principal  base  parameter  analysis  to 
design  of  adaptive  robot  controllers 

93A35554 

SHOWMAN,  G.  L.;  LEAHY,  M.  B.,  JR.  (USAF, 
Inst,  of  Technology,  Wright-Patterson  AFB,  OH)  In: 
1992  IEEE  International  Conference  on  Robotics 
and  Automation,  8th,  Nice,  France,  May  12-14, 

1992,  Proceedings.  Vol.  3  (A93-35501  13-63).  Los 
Alamitos,  CA,  IEEE  Computer  Society  Press,  1992, 
p.  1889-1894. 

The  feasibility  of  using  principal  base  parameter 
analysis  (PBPA)  as  an  aid  in  the  design  and  tuning 
of  adaptive  model-based  controllers  for  industrial 
manipulators  is  investigated.  Results  from  PBPA  are 
utilized  to  select  the  minimal  size  of  the  adaptive 
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parameter  vector  and  to  develop  a  less  heuristic 
procedure  for  controller  tuning.  The  design  proce¬ 
dure  is  illustrated  by  a  simple  two  link  example  and 
then  extended  to  the  first  three  links  of  a 
PUMA-560.  Experimental  analysis  contrasted  with 
an  adaptive  model-based  control  (AMBC)  design 
augmented  with  PBPA  to  a  completely  heuristic 
procedure  used  in  previous  research.  The  incorpo¬ 
ration  of  PBPA  into  the  AMBC  design  minimized 
the  computational  complexity  while  reducing  the 
time  and  expertise  necessary  to  tune  the  controller 
for  satisfactory  tracking  efficacy. 

Developments  of  new  force  reflecting  control 
schemes  and  an  application  to  a  teleopcration 
training  simulator 

93A35545 

KIM,  WON  S.  (JPL,  Pasadena,  CA)  In:  1992  IEEE 
International  Conference  on  Robotics  and  Automa¬ 
tion,  8th,  Nice,  France,  May  12-14,  1992,  Proceed¬ 
ings.  Vol.  2  (A93-35501  13-63).  Los  Alamitos,  CA, 
IEEE  Computer  Society  Press,  1992,  p.  1412-1419. 
Two  schemes  of  force  reflecting  control,  position- 
error  based  force  reflection  and  low-pass-filtered 
force  reflection,  both  combined  with  shared  compli¬ 
ance  control,  were  developed  for  dissimilar 
master-slave  arms.  These  schemes  enabled  high 
force  reflection  gains,  which  were  not  possible  with 
a  conventional  scheme  when  the  slave  arm  was 
much  stiffer  than  the  master  arm.  The  experimental 
results  with  a  peg-in-hole  task  indicated  that  the 
newly  force  reflecting  control  schemes  combined 
with  compliance  control  resulted  in  best  task  perfor¬ 
mances.  As  a  related  application,  a  simulated  force 
reflection/shared  compliance  control  teleoperation 
trainer  was  developed  that  provided  the  operator 
with  the  feel  of  kinesthetic  force  virtual  reality. 


Designing  teleoperator  architectures  for  transparency 

93A35544 

LAWRENCE,  DALE  A.  (Colorado  Univ.,  Boulder) 
In:  1992  IEEE  International  Conference  on  Robotics 
and  Automation,  8th,  Nice,  France,  May  12-14, 
1992,  Proceedings.  Vol.  2  (A93-35501  13-63).  Los 
Alamitos,  CA,  IEEE  Computer  Society  Press,  1992, 
p.  1406-1411. 

The  author  provides  tools  for  analyzing  the  perfor¬ 
mance  of  various  teleoperation  systems,  including 
the  effects  of  communication  delay.  A  general 
multivariable  system  architecture  is  utilized  which 
includes  all  four  types  of  data  transmission  between 
master  and  slave:  force  and  position  in  both  direc¬ 
tions.  It  is  shown  that  a  proper  use  of  all  four  chan¬ 
nels  is  of  critical  importance  in  achieving 
high-performance  telepresence  in  the  sense  of  accu¬ 
rate  transmission  of  task  impedances  to  the  opera¬ 
tor.  Achieved  transparency  and  stability  properties 
of  two  common  architectures,  as  well  as  a 
transparency-optimized  architecture  are  quantita¬ 
tively  compared  on  simplified  one-degree-of- 
freedom  examples. 


Shared  and  traded  telerobotic  visual  control 

93A35529 

PAPANIKOLOPOULOS,  N.  P.;  KHOSLA,  P.  K. 
(Carnegie  Mellon  Univ.,  Pittsburgh,  PA)  In:  1992 
IEEE  International  Conference  on  Robotics  and 
Automation,  8th,  Nice,  France,  May  12-14,  1992, 
Proceedings.  Vol.  1  (A93-35501  13-63).  Los 
Alamitos,  CA,  IEEE  Computer  Society  Press,  1992, 
p.  878-885.  Research  supported  by  DARPA. 

The  authors  address  the  problem  of  integrating  the 
human  operator  with  autonomous  robotic  visual 
tracking  and  servoing  modules.  A  CCD  (charge 
coupled  device)  camera  is  mounted  on  the  end- 
effector  of  a  robot  and  the  task  is  to  servo  around 
a  static  or  moving  rigid  target.  In  manual  control 
mode,  the  human  operator,  with  the  help  of  a 
joystick  and  a  monitor,  commands  robot  motions  in 
order  to  compensate  for  tracking  errors.  In  shared 
control  mode,  the  human  operator  and  the  autono¬ 
mous  visual  tracking  modules  command  motion 
along  orthogonal  sets  of  degrees  of  freedom.  In 
autonomous  control  mode,  the  autonomous  visual 
tracking  modules  are  in  full  control  of  the  servoing 
functions.  Finally,  in  traded  control  mode,  the  con¬ 
trol  can  be  transferred  from  the  autonomous  visual 
modules  to  the  human  operator  and  vice  versa.  The 
authors  present  an  experimental  setup  where  all 
these  different  schemes  have  been  tested.  Experi¬ 
mental  results  of  all  modes  of  operation  are  present¬ 
ed  and  the  related  issues  are  discussed.  In  certain 
degrees  of  freedom  the  autonomous  modules 
perform  better  than  the  human  operator.  On  the 
other  hand,  the  human  operator  can  compensate 
fast  for  failures  in  tracking  while  the  autonomous 
modules  fail. 

An  advanced  teleoperator  control  system  -  Design 
and  evaluation 

93A35526 

LEE,  SUKHAN;  LEE,  HAHK  S.  (JPL,  Pasadena; 
Southern  California  Univ.,  Los  Angeles,  CA); 
(Southern  California  Univ.,  Los  Angeles,  CA)  In: 
1992  IEEE  International  Conference  on  Robotics 
and  Automation,  8th,  Nice,  France,  May  12-14, 
1992,  Proceedings.  Vol.  1  (A93-35501  13-63).  Los 
Alamitos,  CA,  IEEE  Computer  Society  Press,  1992, 
p.  859-864. 

The  design  goal  of  an  advanced  teleoperator  control 
system  is  twofold:  1)  to  allow  the  operator's  manual 
control  to  be  robust  to  system  nonlinearities  such  as 
time  delays  and  operator's  control  errors,  and  2)  to 
support  the  high  performance  of  teleoperation  while 
reducing  the  operator's  control  burden  by  providing 
the  master  and  slave  arms  with  desirable  dynamic 
properties  and  by  allowing  the  slave  arm  to  auto¬ 
matically  perform  such  control  tasks  as  compliance 
and  force  control  in  the  form  of  task  sharing.  The 
authors  present  a  novel  teleoperator  control  system 
achieving  the  above  design  goal  by  taking  the  fol¬ 
lowing  into  consideration:  the  human  dynamics 
involved  in  generating  control  command  based  on 
visual  and  forced  feedback  is  modeled  and  incorpo¬ 
rated  into  the  controller  design  and  evaluation;  the 
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dynamic  characteristics  of  slave  and  master  arms 
are  actively  modified  in  such  a  way  as  to  implement 
the  desirable  dynamic  characteristics;  and  the  force 
feedback  is  redefined  in  terms  of  the  combination  of 
opposition  and  force  discrepancies  in  order  to  estab¬ 
lish  the  required  man/machine  dynamic  coordination 
under  shared  control.  The  proposed  control  system 
with  human  dynamics  in  the  control  loop  is  simulat¬ 
ed  and  compared  with  a  number  of  conventional 
methods  in  the  presence  of  human  control  errors 
and  time  delays. 

A  new  control  scheme  for  bilateral  teleoperating 
systems  -  Lvapunov  stability  analysis 

93A35524 

STRASSBERG,  Y.;  GOLDENBERG,  A.  A.; 

MILLS,  J.  K.  (Toronto  Univ.,  Canada)  In:  1992 
IEEE  International  Conference  on  Robotics  and 
Automation,  8th,  Nice,  France,  May  12-14,  1992, 
Proceedings.  Vol.  1  (A93-35501  13-63).  Los 
Alamitos,  CA,  IEEE  Computer  Society  Press,  1992, 
p.  837-842. 

The  authors  investigate  the  Liapunov  stability  prop¬ 
erty  of  a  control  scheme  for  the  bilateral 
master-slave  teleoperator,  first  introduced  by  the 
authors  in  1990.  Given  the  nominal  models  of  the 
master  and  slave  dynamics,  and  using  an  approxi¬ 
mate  feedback  linearization  control,  based  on  the 
earlier  work  of  M.  W.  Spong  and  M.  Vidyasagar 
(1987),  it  is  shown  that  Liapunov  stability  can  be 
obtained  under  the  assumption  that  the  deviation  of 
the  model  from  the  true  system  satisfies  certain 
norm  inequalities.  From  these  norm  inequalities,  it 
is  shown  that  the  tracking  error  (position/velocity 
and  force/torque)  is  bounded  and  that  sufficient 
conditions  for  Liapunov  stability  can  be  achieved. 
The  control  scheme  is  illustrated  using  the  simula¬ 
tion  of  a  30-degree-of-freedom  master-slave 
teleoperator,  and  the  results  are  presented. 

Real-time  velocity  feedback  obstacle  avoidance  via 
complex  variables  and  conformal  mapping 

93A35515 

MEGHERBI,  DALILA;  WOLOVICH,  W.  A. 

(Brown  Univ.,  Providence,  RI)  In:  1992  IEEE  Inter¬ 
national  Conference  on  Robotics  and  Automation, 
8th,  Nice,  France,  May  12-14,  1992,  Proceedings. 
Vol.  1  (A93-35501  13-63).  Los  Alamitos,  CA,  IEEE 
Computer  Society  Press,  1992,  p.  206-213. 

A  method  for  mobile  robot  motion  planning  is  pre¬ 
sented  for  both  two-  and  three-dimensional  robots. 
For  the  two-dimensional  case,  the  algorithm  is 
based  on  complex  variable  theory  and  conformal 
mapping.  The  method  gives  the  robot  the  capability 
of  avoiding  obstacles  more  efficiently,  smoothly, 
and  at  a  constant  curvilinear  velocity.  In  order  to 
establish  the  structure  of  the  proposed  velocity  feed¬ 
back  obstacle  avoidance  scheme,  several  issues  are 
addressed  and  successfully  resolved.  First,  an  ana¬ 
lytical  solution  is  derived  for  the  case  of  circular 
obstacles.  Second,  the  algorithm  uses  conformal 
mapping  to  establish  the  solution  for  an  arbitrarily 
shaped  obstacle  based  on  the  derived  solution  for  a 


circular  obstacle.  The  use  of  complex  variable 
methodology  with  the  objectives  of  exploiting  the 
powerful  technique  of  uniformal  mapping  consti¬ 
tutes  the  fundamental  characteristic  of  the  proposed 
technique.  Third,  by  analogy,  a  solution  to  3-D 
space-flying  and  subsea  robots  is  also  derived. 

Selecting  distinctive  scene  features  for  landmarks 

93A35503 

LI,  SHIGANG;  TSUJI,  SABURO  (Osaka  Univ., 
Toyonaka,  Japan)  In:  1992  IEEE  International  Con¬ 
ference  on  Robotics  and  Automation,  8th,  Nice, 
France,  May  12-14,  1992,  Proceedings.  Vol.  1.  Los 
Alamitos,  CA,  IEEE  Computer  Society  Press,  1992, 
Autonomous  finding  of  landmarks  for  guiding 
long-distance  navigation  by  a  mobile  is  explored.  In 
a  trial  navigation,  the  robot  continuously  views  and 
memorizes  scenes  along  the  route.  When  the  same 
route  is  subsequently  pursued  again,  the  robot  lo¬ 
cates  and  orients  itself  based  on  the  memorized 
scene.  Since  the  stream  of  images  is  highly  redun¬ 
dant,  it  is  transformed  into  an  intermediate  2(l/2)-D 
representation,  called  the  panoramic  representation 
(PR),  with  a  much  smaller  amount  of  data. 

Although  the  PR  can  be  used  for  guidance  of  the 
autonomous  navigation,  it  still  contains  a  huge 
amount  of  data  for  a  very  long  route.  A  human 
memorizes  only  very  impressive  objects  along  the 
route  and  uses  them  as  landmarks.  The  robot  also 
finds  distinctive  objects  along  the  route  and  memo¬ 
rizes  their  features  as  well  as  spatial  relationships 
for  navigation  guidance.  Three-dimensional  objects 
along  the  route  are  segmented  in  the  PR  by  fusing 
range  estimates  and  color  attributes,  and  then  a 
structure  map  representing  their  arrangement  in 
space  is  obtained.  In  order  to  find  distinctive  objects 
used  for  the  landmarks,  the  spatial  relationships, 
shapes,  and  color  attributes  of  the  objects  are 
examined. 

Adaptive  control  of  robotic  manipulators  using  an 
extended  Kalman  filter 

93A34404 

GOURDEAU,  R.;  SCHWARTZ,  H.  M.  (Ecole 
Polytechnique,  Montreal,  Canada);  (Carleton  Univ., 
Ottawa,  Canada)  ASME,  Transactions,  Journal  of 
Dynamic  Systems,  Measurement,  and  Control  (ISSN 
0022-0434),  vol.  115,  no.  1,  March  1993,  p. 
203-208. 

This  paper  presents  a  new  adaptive  motion  control 
scheme  for  robotic  manipulators.  This  is  an  adaptive 
computed  torque  method  (CTM)  that  requires  only 
position  measurements.  These  measurements  and 
the  input  torques  are  used  in  an  extended  Kalman 
filter  to  estimate  the  inertial  parameters  of  the  full 
non-linear  robot  model  as  well  as  the  joint  positions 
and  velocities.  These  estimates  are  used  by  the 
CTM  to  generate  the  input  torques.  The  theory 
behind  Kalman  filtering  provides  clear  guidelines  on 
the  selection  of  the  design  parameters  for  the  con¬ 
troller  when  noise  is  present.  Simulation  results 
illustrate  the  performance  of  this  scheme  and  dem¬ 
onstrate  its  noise  rejection  properties. 
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A  trigonometric  trajectory  generator  for  robotic 
arms 

93A33776 

SIMON,  DAN;  ISIK,  CAN  (TRW,  Inc.,  Ballistic 
Missiles  Div.,  San  Bernardino,  CA);  (Syracuse 
Univ.,  NY)  International  Journal  of  Control  (ISSN 
0020-7179),  vol.  57,  no.  3,  March  1993,  p.  505-517. 
Interpolation  of  a  robot  joint  trajectory  is  realized 
using  trigonometric  splines.  This  method  is  based 
on  the  assumption  that  joint-space  knots  have  been 
generated  from  Cartesian  knots  by  an  inverse  kine¬ 
matics  algorithm.  The  use  of  trigonometric  splines 
results  in  smooth  joint  trajectories  and  is  amenable 
to  real-time  obstacle  avoidance.  Some  of  the  spline 
parameters  can  be  chosen  to  minimize  an  objective 
function  (e.g.  jerk  or  energy).  If  the  objective  func¬ 
tion  is  minimum  jerk,  a  closed-form  solution  is 
obtained.  This  paper  introduces  a  trajectory  interpo¬ 
lation  algorithm,  discusses  a  method  for  trajectory 
optimization,  and  includes  simulation  examples. 

System  architecture  for  asynchronous 
multi-processor  robotic  control  system 

93A31033 

STEELE,  ROBERT  D.;  LONG,  MARK;  B ACRES, 
PAUL  (JPL,  Pasadena,  CA)  AIAA,  AHS,  and 
ASEE,  Aerospace  Design  Conference,  Irvine,  CA, 
Feb.  16-19,  1993.  8  p. 

The  architecture  for  the  Modular  Telerobot  Task 
Execution  System  (MOTES)  as  implemented  in  the 
Supervisory  Telerobotics  (STELER)  Laboratory  is 
described.  MOTES  is  the  software  component  of 
the  remote  site  of  a  local-remote  telerobotic  system 
which  is  being  developed  for  NASA  for  space  ap¬ 
plications,  in  particular  Space  Station  Freedom 
applications.  The  system  is  being  developed  to 
provide  control  and  supervised  autonomous  control 
to  support  both  space  based  operation  and 
ground-remote  control  with  time  delay.  The 
local-remote  architecture  places  task  planning  re¬ 
sponsibilities  at  the  local  site  and  task  execution 
responsibilities  at  the  remote  site.  This  separation 
allows  the  remote  site  to  be  designed  to  optimize 
task  execution  capability  within  a  limited  computa¬ 
tional  environment  such  as  is  expected  in  flight 
systems.  The  local  site  task  planning  system  could 
be  placed  on  the  ground  where  few  computational 
limitations  are  expected.  MOTES  is  written  in  the 
Ada  programming  language  for  a  multiprocessor 
environment. 

Telerobotic  tasks  requirements  for  planetary 
missions 

93A29146 

THOMAS,  MARIE-CLAUDE;  OCCELLO, 
MICHEL;  TIGLI,  JEAN-YVES  (Nice  Univ., 
Valbonne,  France)  In:  Cooperative  intelligent  robot¬ 
ics  in  space  III;  Proceedings  of  the  Meeting,  Bos¬ 
ton,  MA,  Nov.  16-18,  1992  (A93-29101  10-54). 
Bellingham,  WA,  Society  of  Photo-Optical  Instru¬ 
mentation  Engineers,  1992,  p.  505-513. 

Some  aspects  of  software  design  for  telerobotic 
tasks  are  examined,  with  emphasis  on  spatial  do¬ 


main  with  its  particular  requirements.  A  blackboard 
approach  to  building  specific  architectures  (both 
on-board  and  remote)  is  proposed.  The  design  and 
development  of  a  hybrid  distributed  blackboards 
system  based  on  a  parallel  blackboard  model  are 
discussed.  Two  models  are  presented  (for  static 
decomposition  of  mission  and  dynamic  decision¬ 
making)  which  satisfy  the  same  requirements  of 
software  engineering,  such  as  genericity.  They  adopt 
a  functional  approach  and  emphasize  the  autonomy 
as  a  dynamic  decision-making  criterion.  A  software 
control  architecture  for  space  telerobotics  and  differ¬ 
ent  control  mechanism  filters  are  illustrated. 

Manipulator  control  for  rover  planetary  exploration 

93A29145 

CAMERON,  JONATHAN  M.;  TUNSTEL,  ED¬ 
WARD;  NGUYEN,  TAM;  COOPER,  BRIAN  K. 
(JPL,  Pasadena,  CA)  In:  Cooperative  intelligent 
robotics  in  space  III;  Proceedings  of  the  Meeting, 
Boston,  MA,  Nov.  16-18,  1992  (A93-29101  10-54). 
Bellingham,  WA,  Society  of  Photo-Optica!  Instru¬ 
mentation  Engineers,  1992,  p.  495-504. 

An  anticipated  goal  of  Mars  surface  exploration 
missions  will  be  to  survey  and  sample  surface  rock 
formations  which  appear  scientifically  interesting.  In 
such  a  mission,  a  planetary  rover  would  navigate 
close  to  a  selected  sampling  site  and  the  remote 
operator  would  use  a  manipulator  mounted  on  the 
rover  to  perform  a  sampling  operation.  Techniques 
for  accomplishing  the  necessary  manipulation  for 
the  sampling  components  of  such  a  mission  have 
been  developed  and  are  presented.  We  discuss  the 
implementation  of  a  system  for  controlling  a  seven 
(7)  degree  of  freedom  Puma  manipulator,  equipped 
with  a  special  rock  gripper  mounted  on  a  planetary 
rover  prototype,  intended  for  the  purpose  of  per¬ 
forming  the  sampling  operation.  Control  is  achieved 
by  remote  teleoperation.  This  paper  discusses  the 
real-time  force  control  and  supervisory  control  as¬ 
pects  of  the  rover  manipulation  system.  Integration 
of  the  Puma  manipulator  with  the  existing  distribut¬ 
ed  computer  architecture  is  also  discussed.  The 
work  described  is  a  contribution  toward  achieving 
the  coordinated  manipulation  and  mobility  necessary 
for  a  Mars  sample  acquisition  and  return  scenario. 

Robot  free-flyere  in  space  extravehicular  activity 

93A29141 

WEIGL,  HARALD;  ALEXANDER,  HAROLD  L. 
(MIT,  Cambridge,  MA)  In:  Cooperative  intelligent 
robotics  in  space  III;  Proceedings  of  the  Meeting, 
Boston,  MA,  Nov.  16-18,  1992  (A93-29101  10-54). 
Bellingham,  WA,  Society  of  Photo-Optical  Instru¬ 
mentation  Engineers,  1992,  p.  458-469. 

Attention  is  given  to  the  development  of  a  remote 
robot  with  maneuverability  and  dexterity  compara¬ 
ble  to  that  of  a  space-suited  astronaut  with  a 
manned  maneuvering  unit  capable- of  handling  many 
of  the  tasks  currently  planned  for  astronauts  during 
EVA.  A  real-time  vision-based  navigation  and  con¬ 
trol  system  for  an  underwater  space  robot  simulator, 
the  Submersible  for  Telerobotic  and  Astronautical 
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Research  (STAR)  is  examined.  The  system,  imple¬ 
mented  with  standard,  inexpensive  computer  hard¬ 
ware,  exhibits  excellent  performance  and  robustness 
characteristics  for  a  variety  of  applications,  includ¬ 
ing  automatic  station-keeping  and  large  controlled 
maneuvers.  Experimental  results  are  presented 
which  indicate  the  precision,  accuracy,  and  robust¬ 
ness  to  disturbances  of  the  vision-based  control 
system.  The  study  proves  the  feasibility  of  using 
vision-based  control  and  navigation  for  remote  ro¬ 
bots  and  provides  a  foundation  for  developing  a 
system  for  general  space  robot  tasks. 

Intelligent  virtual  interfaces  for  telerobotics 

93A29136 

GRIN STEIN,  GEORGES  G.;  MAYBURY,  MARK 
T.;  MITCHELL,  RICHARD  B.  (Mitre  Corp.,  Bed¬ 
ford,  MA)  In:  Cooperative  intelligent  robotics  in 
space  III;  Proceedings  of  the  Meeting,  Boston,  MA, 
Nov.  16-18,  1992  (A93-29101  10-54).  Bellingham, 
WA,  Society  of  Photo-Optical  Instrumentation  Engi¬ 
neers,  1992,  p.  401-408. 

Research  in  two  key  areas  of  intelligent  interfaces 
to  support  teleoperation  -  intelligent  virtual  inter¬ 
faces  and  plan-based  communication  -  is  described. 
The  former  promises  a  visual,  auditory,  and  tactile 
experience,  and  the  latter  promises  more  natural  and 
effective  communications.  Ways  in  which  these 
capabilities  might  fit  into  a  virtual  reality  interface 
for  teleoperation  are  shown.  A  device  integrator 
couples  physical  (e.g.,  dataglobe  and  eye-tracker) 
and  linguistic  input,  which  is  then  interpreted  by  a 
plan  recognizer  which  would  interpret  both  the 
communicative  action  as  well  as  physical  action 
intended  by  the  operator.  Once  an  action  was  per¬ 
formed  by  the  remote  physical  or  simulated  (for 
training)  robot,  the  results  would  be  communicated 
to  the  user  via  a  variety  of  modes,  including  visual, 
auditory,  and  tactile.  These  components  need  to  be 
integrated  and  then  evaluated. 

Teleprogramming  a  cooperative  space  robotic 
workcell  for  Space  Station 

93A29109 

HAULE,  D.  D.;  NOORHOSSEINI,  S.  M.; 
MALOWANY,  A.  S.  (McGill  Univ.,  Montreal, 
Canada)  In:  Cooperative  intelligent  robotics  in  space 
III;  Proceedings  of  the  Meeting,  Boston,  MA,  Nov. 
16-18,  1992  (A93-29101  10-54).  Bellingham,  WA, 
Society  of  Photo-Optical  Instrumentation  Engineers, 
1992,  p.  102-116. 

Robotics  and  automation  in  remote  hostile 
enviroments  such  as  space  (planet)  exploration,  for 
which  working  conditions  -  environment  model  and 
robot  operating  conditions  -  are  unknown  or  very 
partially  known,  are  studied.  Several  interrelated 
topics  for  Space  Station  automation  using  a 
teleprogrammable  space  robotic  workcell  (SRW)  are 
discussed.  The  operation  rationale  for  SRW  is  to 
free  up  crew  time  with  the  ultimate  goal  of  making 
on-board  crew  involvement  in  SRW  tasks  optional, 
while  solving  the  problem  of  'automation  of  opera¬ 
tor  or  supervisory  control'.  The  key  issues  of  task 


level  teleprogramming  as  an  attribute  for  operating 
and  decisional  autonomy  vs  the  concepts  of  classi¬ 
cal  teleoperation  and  telerobotics  are  also  addressed. 

Flight  Telerobotic  Servicer  legacy 

93A29106 

SHATTUCK,  PAUL  L.;  LOWRIE,  JAMES  W. 
(Martin  Marietta  Astronautics  Group,  Denver,  CO) 
In:  Cooperative  intelligent  robotics  in  space  III; 
Proceedings  of  the  Meeting,  Boston,  MA,  Nov. 
16-18,  1992  (A93-29101  10-54).  Bellingham,  WA, 
Society  of  Photo-Optical  Instrumentation  Engineers, 
1992,  p.  60-74. 

The  technology  evolution  that  stemmed  from  devel¬ 
oping  and  integrating  a  dexterous  robot  into  a 
manned  system,  the  Space  Shuttle,  is  traced.  Em¬ 
phasis  is  placed  on  the  safety  and  reliability  require¬ 
ments  for  a  man-rated  system  as  the  critical  factors 
which  drive  the  overall  system  architecture.  Task 
requirements  and  operational  concepts  for  servicing 
and  maintenance  of  space  platforms,  origins  of 
technology  for  dexterous  robotic  systems,  issues 
associated  with  space  qualification  of  components, 
and  development  of  the  industrial  base  to  support 
space  robotics  are  also  discussed.  The  Flight 
Telerobotic  Servicer  (FTS),  developed  to  enhance 
and  provide  a  safe  alternative  to  the  human  pres¬ 
ence  in  space,  had  completed  the  major  component 
development  activities  for  the  flight  system  at  the 
point  of  termination.  The  FTS  Technology  Capture 
Program  provides  a  mechanism  for  transfering  the 
component  technologies  to  the  user  community  and 
could  serve  as  a  focal  point  for  the  A&R  program 
thrust  in  on-orbit  servicing. 

Real  time  proximity  cues  for  teleoperation  using 
model  based  force  reflection 

93A27033 

BRUNO,  GUY;  MORGENTHALER,  MATTHEW 
K.  (Martin  Marietta  Civil  Space  and  Communica¬ 
tions  Co.,  Denver,  CO)  In:  Cooperative  intelligent 
robotics  in  space  II;  Proceedings  of  the  Meeting, 
Boston,  MA,  Nov.  12-14,  1991  (A93-27001  09-54). 
Bellingham,  WA,  Society  of  Photo-Optical  Instru¬ 
mentation  Engineers,  1992,  p.  346-355. 

The  problem  of  providing  model  based  proximity 
cues  using  force  reflection  for  teleoperation  under 
time  delay  is  addressed.  A  novel  use  of  artificial 
potential  fields  is  proposed  as  a  teleoperator  aid  to 
efficiently  provide  a  predictive  tactile  display.  Sev¬ 
eral  new  artificial  potential  models  are  presented 
which  are  used  to  convey  accurate  shape  and  prox¬ 
imity  information  by  generating  handcontroller 
forces  based  on  the  potential  gradient.  These  new 
potential  gradients  are  shown  to  have  an  efficient 
implementation  via  exact  computation  as  a  neural 
network.  A  real  time  prototype  implementation  and 
integration  with  Martin  Marietta's  teleautonomous 
testbed,  is  discussed.  Evaluations  are  made  with 
human  operators  performing  tasks  using  industrial 
manipulators  under  time  delay  scenarios. 
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Interactive  Scene  Analysis  Module  - 
A  sensor-database  fusion  system  for  telerobotic 
environments 

93A27032 

COOPER,  ERIC  G.;  VAZQUEZ,  SIXTO  L.; 
GOODE,  PLESENT  W.  (NASA,  Langley  Research 
Center,  Hampton,  VA)  In:  Cooperative  intelligent 
robotics  in  space  II;  Proceedings  of  the  Meeting, 
Boston,  MA,  Nov.  12-14,  1991  (A93-27001  09-54). 
Bellingham,  WA,  Society  of  Photo-Optical  Instru¬ 
mentation  Engineers,  1992,  p.  334-345. 
Accomplishing  a  task  with  telerobotics  typically 
involves  a  combination  of  operator  control/ 
supervision  and  a  'script'  of  preprogrammed  com¬ 
mands.  These  commands  usually  assume  that  the 
location  of  various  objects  in  the  task  space  con¬ 
form  to  some  internal  representation  (database)  of 
that  task  space.  The  ability  to  quickly  and  accurate¬ 
ly  verify  the  task  environment  against  the  internal 
database  would  improve  the  robustness  of  these 
preprogrammed  commands.  In  addition,  the  on-line 
initialization  and  maintenance  of  a  task  space  data¬ 
base  is  difficult  for  operators  using  Cartesian  coor¬ 
dinates  alone.  This  paper  describes  the  Interactive 
Scene’  Analysis  Module  (ISAM)  developed  to  pro¬ 
vide  taskspace  database  initialization  and  verifica¬ 
tion  utilizing  3-D  graphic  overlay  modelling,  video 
imaging,  and  laser  radar  based  range  imaging. 
Through  the  fusion  of  taskspace  database  informa¬ 
tion  and  image  sensor  data,  a  verifiable  taskspace 
model  is  generated  providing  location  and  orienta¬ 
tion  data  for  objects  in  a  task  space.  This  paper  also 
describes  applications  of  the  ISAM  in  the  Intelligent 
Systems  Research  Laboratory  (ISRL)  at  NASA 
Langley  Research  Center,  and  discusses  its  perfor¬ 
mance  relative  to  representation  accuracy  and  opera¬ 
tor  interface  efficiency. 


Incorporating  robot  vision  in  tele-autonomous 

systems 

93A27031 

LEJUN,  SHAO;  VOLZ,  RICHARD;  CONWAY, 
LYNN;  WALKER,  M.  W.  (Nanyang  Technological 
Univ.,  Singapore);  (Texas  A  &  M  Univ.,  College 
Station);  (Michigan  Univ.,  Ann  Arbor)  In:  Coopera¬ 
tive  intelligent  robotics  in  space  II;  Proceedings  of 
the  Meeting,  Boston,  MA,  Nov.  12-14,  1991 
(A93-27001  09-54).  Bellingham,  WA,  Society  of 
Photo-Optical  Instrumentation  Engineers,  1992,  p. 
323-333. 

A  new  method  is  proposed  to  tele-control  the  move¬ 
ment  of  a  remote  robot  when  the  movement  in¬ 
volves  contact  with  objects  and  when  the  control 
involves  a  significant  time  delay.  In  this  method,  a 
vision  system  is  incorporated  into  the  tele-autono¬ 
mous  systems.  The  vision  system  is  used  to  do 
vision  sensory  information  feedback  to  update  local 
world  model  and  to  implement  a  relative  move 
mode  to  control  the  remote  robot.  This  method  will 
effectively  overcome  some  of  the  limitations  of 
current  tele-robot  control  systems. 


A  telerobotic  virtual  control  system 

93A27030 

ZHAI,  SHUMIN;  MILGRAM,  PAUL  (Toronto 
Univ.,  Canada)  In:  Cooperative  intelligent  robotics 
in  space  II;  Proceedings  of  the  Meeting,  Boston, 

MA,  Nov.  12-14,  1991  (A93-27001  09-54). 
Bellingham,  WA,  Society  of  Photo-Optical  Instru¬ 
mentation  Engineers,  1992,  p.  311-322.  Research 
supported  by  Defence  and  Civil  Inst,  of  Environ¬ 
mental  Medicine. 

A  project  to  develop  a  telerobotic  virtual  control 
capability,  currently  underway  at  the  University  of 
Toronto,  is  described.  The  project  centers  on  a  new 
mode  of  interactive  telerobotic  control  based  on  the 
technology  of  combining  computer  generated 
stereographic  images  with  remotely  transmitted 
stereoscopic  video  images.  A  virtual  measurement 
technique,  in  conjunction  with  a  basic  level  of  digi¬ 
tal  image  processing,  comprising  zooming,  parallax 
adjustment,  edge  enhancement,  and  edge  detection, 
have  been  developed  to  assist  the  human  operator  in 
visualization  of  the  remote  environment  and  in 
spatial  reasoning.  The  aim  is  to  maintain  target 
recognition,  tactical  planning  and  high  level  control 
functions  in  the  hands  of  the  human  operator,  with 
the  computer  performing  low  level  computation  and 
control.  Control  commands  initiated  by  the  operator 
are  implemented  through  manipulation  of  a  virtual 
image  of  the  robot  system,  merged  with  a  live  video 
image  of  the  remote  scene.  This  paper  discusses  the 
philosophy  and  objectives  of  the  project,  with  em¬ 
phasis  on  the  underlying  human  factors  consider¬ 
ations  in  the  design,  and  reports  the  progress  made 
to  date  in  this  effort. 

World  model  and  its  uncertainty  in  supervisory 
robot  control 

93A27027 

PARK,  JONG  H.;  SHERIDAN,  THOMAS  B.  (MIT, 
Cambridge,  MA)  In:  Cooperative  intelligent  robotics 
in  space  II;  Proceedings  of  the  Meeting,  Boston, 
MA,  Nov.  12-14,  1991  (A93-27001  09-54). 
Bellingham,  WA,  Society  of  Photo-Optical  Instru¬ 
mentation  Engineers,  1992,  p.  278-288. 

This  paper  describes  new  methods  to  deal  with 
uncertainty  in  the  position  and  orientation  of  objects 
in  the  world  model  in  the  context  of  robot 
teleoperations.  The  virtual  obstacle  (object)  is  de¬ 
fined  to  represent  objects  with  uncertainty  bounds, 
which  are  constructed  by  the  operator,  who  uses 
geometric  data  base  and  a  6  d.o.f.  input  device, 
while  viewing  video  displays.  These  virtual  obsta¬ 
cles  are  updated  as  the  cameras  move.  Also  a  new 
method  to  build  the  world  model  by  so  called 
'flying-and-matching'  is  introduced.  Experiments 
have  been  performed  with  human  subjects  to  evalu¬ 
ate  the  proposed  schemes. 

Using  qualitative  reasoning  for  robot  task  planning 

93A27015 

SCHAEFER,  PHIL  (Martin  Marietta  Advanced 
Computing  Technology,  Denver,  CO)  In:  Coopera¬ 
tive  intelligent  robotics  in  space  II;  Proceedings  of 
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the  Meeting,  Boston,  MA,  Nov.  12-14,  1991 
(A93-27001  09-54).  Bellingham,  WA,  Society  of 
Photo-Optical  Instrumentation  Engineers,  1992,  p. 
139-148. 

Application  of  qualitative  reasoning  (QR)  tech¬ 
niques  to  describe  reasoning  tasks  for  different 
classes  of  complex  systems  is  discussed.  It  is  shown 
that  effective  task  planning  for  robotic  systems  can 
be  performed  using  a  qualitative,  model-based  para¬ 
digm.  This  approach  has  certain  advantages  over  the 
usual  operator-based  techniques.  The  robotics 
workspace  can  be  modeled  modularly.  Nonlinear 
methods  and  the  truth  maintenance  system  make  the 
planning  efficient,  which  allows  complex  search 
spaces  to  be  searched  with  reasonable  response 
time.  QR  representation  for  robot  task  planning 
using  the  Multi  Purpose  Causal  (MPC)  software 
tool  can  be  used  for  exception  detection,  contingen¬ 
cy  identification,  and  contingency  recovery.  It  is 
concluded  that  the  major  advantage  of  the  QR  ap¬ 
proach  is  that  all  of  the  knowledge  to  perform  each 
of  the  planning  and  failure  management  tasks  is 
contained  in  one  model. 

Time-optimal  trajectory  generation  for  coordinated 
robotic  manipulators  using  cell-to-cell  mapping 
method 

93A27013 

WANG,  FEI-YUE;  PU,  BING  (Arizona  Univ., 
Tucson)  In:  Cooperative  intelligent  robotics  in  space 
II;  Proceedings  of  the  Meeting,  Boston,  MA,  Nov. 
12-14,  1991  (A93-27001  09-54).  Bellingham,  WA, 
Society  of  Photo-Optical  Instrumentation  Engineers, 
1992,  p.  115-122. 

A  cell-to-cell  mapping  method  is  used  to  solve  the 
problem  of  optimal  trajectory  generation  for  coor¬ 
dinated  robotic  manipulators  handling  a  common 
object  along  specified  geometric  paths.  The  method 
is  based  on  the  cell  state  concept  in  conjunction 
with  discretized  controls  and  cost  functions.  A  hier¬ 
archical  search  algorithm  which  makes  it  possible  to 
implement  parallel  computation  and  to  reduce  the 
computation  time  is  proposed. 

Planning  and  executing  visually  constrained  robot 
motions 

93A2701 1 

FOX,  ARMANDO;  CASTANO,  ANDRES; 
HUTCHINSON,  SETH  (Illinois  Univ.,  Urbana)  In: 
Cooperative  intelligent  robotics  in  space  II;  Pro¬ 
ceedings  of  the  Meeting,  Boston,  MA,  Nov.  12-14, 
1991  (A93-27001  09-54).  Bellingham,  WA,  Society 
of  Photo-Optical  Instrumentation  Engineers,  1992, 
p.  90-97. 

Despite  progress  in  visual  servo  control  of  robot 
motions,  to  date  the  corresponding  motion  planning 
problem  has  not  been  investigated.  In  this  paper,  we 
present  an  implemented  planner  for  the  special  case 
of  a  polyhedral  world,  extending  previous  preimage 
type  planners  to  exploit  visual  constraint  surfaces  in 
a  fixed-camera  robotic  system  featuring  closed-loop 
visual  servo  control.  We  present  the  mathematics  of 
a  hybrid  (visual/position  feedback)  resolved-rate 


motion  control  strategy  for  executing  these  plans, 
featuring  projection  equations  defined  solely  in 
terms  of  a  small  set  of  observable  parameters  that 
are  directly  obtained  from  our  calibration  process. 
We  conclude  with  experimental  results,  a  descrip¬ 
tion  of  ongoing  research  and  the  contribution  of  our 
work  to  date. 

Controller  design  for  a  force-reflecting  teleopcrator 
system  with  kinematically  dissimilar  master  and 
slave 

93A23844 

JANSEN,  J.  F.;  KRESS,  R.  L.;  BABCOCK,  S.  M. 
(Oak  Ridge  National  Lab.,  TN)  ASME,  Transac¬ 
tions,  Journal  of  Dynamic  Systems,  Measurement, 
and  Control  (ISSN  0022-0434),  vol.  114,  no.  4, 

Dec.  1992,  p.  641-649.  Research  sponsored  by 
USAF. 

The  purpose  of  this  paper  is  to  develop  a  controller 
for  a  force-reflecting  teleoperator  system  having 
kinematically  dissimilar  master  and  slave.  The  con¬ 
troller  is  a  stiffness  controller  for  both  the  master 
and  slave.  A  mathematical  problem  associated  with 
representing  orientations  using  Euler  angles  is  de¬ 
scribed,  and  Euler  parameters  are  proposed  as  a 
solution.  The  basic  properties  of  Euler  parameters 
are  presented,  specifically  those  pertaining  to  stiff¬ 
ness  control.  The  stiffness  controller  for  both  the 
master  and  the  slave  is  formulated  using  Euler  pa¬ 
rameters  to  represent  orientation  and  a  Liapunov 
stability  proof  is  presented  for  the  controller.  The 
master  portion  of  the  control  scheme  is  implement¬ 
ed  on  a  six-degree-of-freedom  master. 

Space  based  robot  manipulators  -  Dynamics  of 
contact  and  trajectory  planning  for  impact 
minimization 

93A22827 

WEE,  LIANG-BOON;  WALKER,  MICHAEL  W. 
(Michigan  Univ.,  Ann  Arbor)  In:  1992  American 
Control  Conference,  1 1th,  Chicago,  IL,  June  24-26, 
1992,  Proceedings.  Vol.  1  (A93-22776  07-63). 
Piscataway,  NJ,  Institute  of  Electrical  and  Electron¬ 
ics  Engineers,  1992,  p.  771-775. 

The  authors  consider  contact  between  free-flying 
space  robots,  and  the  minimization  of  the  impulse  at 
impact.  They  begin  with  a  presentation  of  a  model 
of  space  robot  which  takes  into  account  external 
applied  forces.  A  contact  model  which  considers 
both  linear  and  angular  motion  between  contacting 
systems  is  presented.  Two  approaches  for  trajectory 
planning  in  Cartesian  space  are  discussed,  and  a 
strategy  for  achieving  both  the  primary  objective  of 
trajectory  tracking  in  Cartesian  space  and  the  sec¬ 
ondary  objective  of  impact  minimization  through 
configuration  space  planning  is  presented.  The  strat¬ 
egy  was  tested  on  a  15  degree-of-freedom  space 
robot,  and  simulation  results  are  presented. 

Research  state  of  the  art  of  mobile  robots  in  China 

93A19109 

WU,  LIN;  ZHAO,  JINGLUN;  ZHANG,  PENG;  LI, 
SHIQING  (Harbin  Inst,  of  Technology,  China); 
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(Shenyang  Inst,  of  Automation,  China);  (Changsha 
Inst,  of  Technology,  China);  (Shenyang  Inst,  of 
Automation,  China)  In:  Mobile  robots  V;  Proceed¬ 
ings  of  the  Meeting,  Boston,  MA,  Nov.  8,  9,  1990 
(A93- 19070  05-63).  Bellingham,  WA,  Society  of 
Photo-Optical  Instrumentation  Engineers,  1991,  p. 
598-601. 

Mobile  robots  developed  in  China  for  applications 
in  dangerous  environments  are  briefly  described. 
They  include  a  master-slave  telerobot,  a  six-legged 
robot,  a  remote  inspection  robot  for  wall  surface 
travelling,  a  versatile  crawler  moving  robot,  and  an 
autonomous  mobile  robot. 

Plan-behavior  interaction  in  autonomous  navigation 

93A19100 

LIM,  WILLIE;  EILBERT,  JIM  (Grumman  Corpo¬ 
rate  Research  Center,  Bethpage,  NY)  In:  Mobile 
robots  V;  Proceedings  of  the  Meeting,  Boston,  MA, 
Nov.  8,  9,  1990  (A93- 19070  05-63).  Bellingham, 
WA,  Society  of  Photo-Optical  Instrumentation  Engi¬ 
neers,  1991,  p.  464-475. 

A  scheme  for  plan-behavior  interaction  in  a  reactive 
robot  for  navigating  in  an  indoor  environment  is 
presented  which  is  based  on  relating  behaviors  to 
higher  level  plans.  This  approach  makes  possible 
the  interaction  between  reactive  behaviors  and  more 
explicit  and  deliberate  reasoning  processes.  The 
capability  of  subsumption  based  robots  has  been 
extended  by  allowing  higher  level  reasoning  pro¬ 
cesses  to  tune  the  selection  and  activation  of  behav¬ 
iors.  The  robot's  overall  behavior  can  be  specialized 
and  optimized  to  the  specific  goal  or  mission.  Par¬ 
ticular  attention  is  given  to  the  multiagent  architec¬ 
ture  for  script-based  navigation  being  developed  for 
a  robot  called  SmartyCat. 

Environment  model  for  mobile  robots  indoor 
navigation 

93A19099 

ROTH-TABAK,  YUVAL;  WEYMOUTH,  TERRY 
E.  (Michigan  Univ.,  Ann  Arbor)  In:  Mobile  robots 
V;  Proceedings  of  the  Meeting,  Boston,  MA,  Nov. 

8,  9,  1990  (A93-19070  05-63).  Bellingham,  WA, 
Society  of  Photo-Optical  Instrumentation  Engineers, 
1991,  p.  453-463. 

An  autonomous  mobile  robot  must  be  able  to  com¬ 
bine  uncertain  sensory  information  with  prior 
knowledge  of  the  world.  Moreover,  these  operations 
have  to  be  performed  fast  enough  to  be  able  to  react 
to  the  changes  in  the  world.  This  paper  presents  a 
model-driven  system  for  a  real-time  indoor  mobile 
robot.  As  the  robot  is  constantly  in  motion,  informa¬ 
tion  from  an  Environment  Model  is  used  to  antici¬ 
pate  information-rich  features  and  to  direct  selective 
sensing.  Uncertain  sensor  information  is  combined 
with  prior  World  Model  knowledge  to  reduce  uncer¬ 
tainty,  and  the  remaining  uncertainty  is  directly 
represented  by  flexible  ranges  of  values.  We  present 
a  hall-following  robot,  based  on  this  system,  which 
exhibits  real-time  navigation  performance.  It  does 
this  despite  primitive  and  relatively  slow  sensing, 
motor  control,  and  communications  capabilities. 


This  system  combines  sensing,  action,  and  cogni¬ 
tion,  which  are  the  major  building  blocks  for  any 
autonomous  system. 

Coping  with  complexity  in  the  navigation  of  an 
autonomous  mobile  robot 

93A19098 

DODDS,  DAVID  R.  In:  Mobile  robots  V;  Proceed¬ 
ings  of  the  Meeting,  Boston,  MA,  Nov.  8,  9,  1990 
(A93-19070  05-63).  Bellingham,  WA,  Society  of 
Photo-Optical  Instrumentation  Engineers,  1991,  p. 
448-452. 

This  paper  discusses  the  integrated  use  of  dynamic 
systems  theory,  neural  networks  and  evolutionary 
programming  methods  as  a  means  of  coping  with 
complexity  in  automatic  plan  generation.  Plan  ele¬ 
ments  representing  actions  are  mapped  into 
phase-space  and  are  examined  for  stability  by 
searching  for  and  identifying  any  'attractors'.  A 
knowledge-based  system  for  doing  this  is  described. 
As  a  means  of  coping  with  unexpected  environ¬ 
ments  modifications  of  plans  are  made  by  the  plan¬ 
ning  system  using  an  evolutionary  programming 
method  coupled  with  a  neural  network  approach. 

Towards  a  versatile  control  system  for  mobile 
robots 

93A19092 

NOREILS,  FABRICE  R.  (Carnegie  Mellon  Univ., 
Pittsburgh,  PA)  In:  Mobile  robots  V;  Proceedings  of 
the  Meeting,  Boston,  MA,  Nov.  8,  9,  1990 
(A93- 19070  05-63).  Bellingham,  WA,  Society  of 
Photo-Optical  Instrumentation  Engineers,  1991, 
p.  384-396. 

The  goal  of  a  mobile  robot  is  not  only  to  perform  a 
large  variety  of  tasks  but  also  to  adapt  its  behavior 
to  the  dynamic  of  the  environment.  Such  a  robot 
needs  both  reflexive  and  planning  capabilities. 
Moreover,  we  wish  that  a  mobile  robot  be  able  to 
coordinate  its  activity  with  other  mobile  robots.  This 
paper  is  intended  to  show  that  the  integration  of  all 
these  capabilities  (reflexivity,  planning,  and  coordi¬ 
nation)  is  feasible;  in  order  to  achieve  this  goal,  a 
new  robot  architecture  is  proposed.  This  architecture 
is  composed  of  three  levels:  functional,  control,  and 
planner.  Furthermore,  this  architecture  provides 
important  features  such  as  a  progressive  and  pro¬ 
grammable  reactivity,  robustness,  additivity,  and 
versatility.  This  paper  emphasizes  the  control  level 
and  how  it  is  used  by  the  planner  level.  In  order  to 
demonstrate  the  feasibility  of  such  an  approach,  a 
complete  implementation  of  this  architecture  on  our 
mobile  robot,  including  experiments,  are  described. 

Intelligent  piloting  tools  for  control  of  an 
autonomous  mobile  robot 

93A19091 

MALOTAUX,  E.;  ALIMENTI,  R.;  BOGAERT,  M.; 
GASP  ART,  P.  (Centre  de  Recherches  Scientifiques 
et  Techniques  de  l'Industrie  des  Fabrications 
Mecaniques,  Brussels,  Belgium)  In:  Mobile  robots 
V;  Proceedings  of  the  Meeting,  Boston,  MA,  Nov. 

8,  9,  1990  (A93- 19070  05-63).  Bellingham,  WA, 
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Society  of  Photo-Optical  Instrumentation  Engineers, 
1991,  p.  372-383.  Research  supported  by  IRSIA, 
ACEC,  Elbicon,  et  al. 

Piloting  procedures  based  on  a  pilot  control  algo¬ 
rithm  (Autopil)  are  described  including  an  off-line 
graph-search  path  planner  (Pilastar)  and  on-line 
constrained  piloting  (Pilcpp).  This  very  precise, 
flexible  and  inexpensive  in  computation  time  path 
execution  control  algorithm  produces  natural  trajec¬ 
tories  and  takes  into  account  all  the  physical  con¬ 
straints  on  speed  and  acceleration. 

Hybrid  navigational  control  scheme  for  autonomous 
platforms 

93A19086 

HOLLAND,  JOHN;  EVERETT,  H.  R.; 
GILBREATH,  G.  A.  (Cybermotion,  Inc.,  Roanoke, 
VA);  (U.S.  Navy,  Naval  Ocean  Systems  Center,  San 
Diego,  CA)  In:  Mobile  robots  V;  Proceedings  of  the 
Meeting,  Boston,  MA,  Nov.  8,  9,  1990  (A93-19070 
05-63).  Bellingham,  WA,  Society  of  Photo-Optical 
Instrumentation  Engineers,  1991,  p.  291-298. 

A  hybrid  navigational  scheme  capable  of  periodical¬ 
ly  resetting  the  heading  and  position  of  the  robot 
using  a  priori  knowledge  of  navigational  cues  in  the 
environment  is  presented.  The  scheme  integrates  the 
desired  features  of  guidepath  following,  unrestricted 
path  planning,  and  virtual  path  navigation  into  a 
robust  navigational  package  better  able  to  cope  with 
the  varied  demands  of  real-world  operation.  The 
path  planner  uses  fixed  guidepaths  and  pre¬ 
programmed  paths  whenever  possible,  but  maintains 
the  ability  to  roam  freely  about  the  workspace. 

Impact  of  uncertain  terrain  models  on  the  weighted 
region  problem 

93A19084 

MOBASSERI,  BIJAN  G.  (Villanova  Univ.,  PA)  In: 
Mobile  robots  V;  Proceedings  of  the  Meeting,  Bos¬ 
ton,  MA,  Nov.  8,  9,  1990  (A93-19070  05-63). 
Bellingham,  WA,  Society  of  Photo-Optical  Instru¬ 
mentation  Engineers,  1991,  p.  270-277. 

Outdoor  navigation  is  characterized  by  motions 
through  regions  of  varied  terrain.  The  weighted 
region  problem  (WRP),  is  a  generalization  of  the 
obstacle  avoidance  problem  with  1/infinity  cost 
structure.  By  assigning  indices  to  surface  patches 
proportional  to  their  traversability,  WRP  seeks  a 
path  with  the  shortest  length,  in  the  weighted  sense. 
This  work  generalizes  the  WRP  paradigm  by  stating 
that  the  traversability  indices  may  only  be  available 
through  a  probability  distribution.  The  reported 
indices,  therefore,  are  not  an  exact  description  of 
the  terrain,  rather,  they  are  an  observation  drawn 
from  their  respective  distributions.  Development  of 
a  decision  basis  directing  the  search  is  an  objective 
of  this  paper. 

Safe  motion  planning  for  mobile  agents  -  A  model 
of  reactive  planning  for  multiple  mobile  agents 

93A19083 

FUJIMURA,  KIKUO  (Oak  Ridge  National  Lab., 

TN)  In:  Mobile  robots  V;  Proceedings  of  the  Meet¬ 


ing,  Boston,  MA,  Nov.  8,  9,  1990  (A93-19070 
05-63).  Bellingham,  WA,  Society  of  Photo-Optical 
Instrumentation  Engineers,  1991,  p.  260-269. 

The  problem  of  motion  planning  for  multiple  mo¬ 
bile  agents  is  studied.  Each  planning  agent  inde¬ 
pendently  plans  its  own  action  based  on  its  map 
which  contains  a  limited  information  about  the 
environment.  In  an  environment  where  more  than 
one  mobile  agent  interacts,  the  motions  of  the  ro¬ 
bots  are  uncertain  and  dynamic.  A  model  for  reac¬ 
tive  agents  is  described  and  simulation  results  are 
presented  to  show  their  behavior  patterns. 

Terrain  classification  in  navigation  of  an 
autonomous  mobile  robot 

93A19076 

DODDS,  DAVID  R.  In:  Mobile  robots  V;  Proceed¬ 
ings  of  the  Meeting,  Boston,  MA,  Nov.  8,  9,  1990 
(A93-19070  05-63).  Bellingham,  WA,  Society  of 
Photo-Optical  Instrumentation  Engineers,  1991, 
p.  82-89. 

In  this  paper  we  describe  a  method  of  path  planning 
that  integrates  terrain  classification  (by  means  of 
fractals),  the  certainty  grid  method  of  spatial  repre¬ 
sentation,  Kehtamavaz  Griswold  collision-zones, 
Dubois  Prade  fuzzy  temporal  and  spatial  knowledge, 
and  non-point  sized  qualitative  navigational  plan¬ 
ning.  An  initially  planned  ('end-to-end')  path  is 
piece-wise  modified  to  accommodate  known  and 
inferred  moving  obstacles,  and  includes  attention  to 
time-varying  multiple  subgoals  which  may  influence 
a  section  of  path  at  a  time  after  the  robot  has  begun 
traversing  that  planned  path. 

Real-time  map-building  for  fast  mobile  robot 
obstacle  avoidance 

93A19075 

BORENSTEIN,  JOHANN;  KOREN,  YORAM 
(Michigan  Univ.,  Ann  Arbor)  In:  Mobile  robots  V; 
Proceedings  of  the  Meeting,  Boston,  MA,  Nov.  8,  9, 
1990  (A93-19070  05-63).  Bellingham,  WA,  Society 
of  Photo-Optical  Instrumentation  Engineers,  1991, 
p.  74-81. 

This  paper  introduces  HIMM  (histogramic 
in-motion  mapping),  a  new  method  for  real-time 
map  building  with  a  mobile  robot  in  motion.  HIMM 
represents  data  in  a  two-dimensional  array  (called  a 
histogram  grid)  that  is  updated  through  rapid  con¬ 
tinuous  sampling  of  the  onboard  range  sensors  dur¬ 
ing  motion.  Rapid  in-motion  sampling  results  in  a 
statistical  map  representation  that  is  well-suited  to 
modeling  inaccurate  and  noisy  range-sensor  data. 
HIMM  is  integral  part  of  an  obstacle  avoidance 
algorithm  and  allows  the  robot  to  immediately  use 
the  mapped  information  in  real-time  obstacle-avoid¬ 
ance.  The  benefits  of  this  integrated  approach  are 
twofold:  (1)  quick,  accurate  mapping;  and  (2)  safe 
navigation  of  the  robot  toward  a  given  target. 

HIMM  has  been  implemented  and  tested  on  a  mo¬ 
bile  robot.  Its  dual  functionality  was  demonstrated 
through  numerous  tests  in  which  maps  of  unknown 
obstacle  courses  were  created,  while  the  robot 
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simultaneousy  performed  real-time  obstacle  avoid¬ 
ance  maneuvers  at  speeds  of  up  to  0.78  m/sec. 

Vehicle  path  planning  via  dual  world 
representations 

93A19072 

PECK,  ALEX;  BREUL,  HARRY  (Grumman  Cor¬ 
porate  Research  Center,  Bethpage,  NY)  In:  Mobile 
robots  V;  Proceedings  of  the  Meeting,  Boston,  MA, 
Nov.  8,  9,  1990  (A93-19070  05-63).  Bellingham, 
WA,  Society  of  Photo-Optical  Instrumentation  Engi¬ 
neers,  1991,  p.  30-38. 

A  technique  is  developed  whereby  a  mobile  robot 
equipped  with  sonar  sensors  autonomously  explores 
a  hallway  environment,  and  during  exploration  dy¬ 
namically  builds  two  types  of  maps:  a  graph  of 
places  defined  by  distinctive  sonar  events,  and  a 
grid  map  from  dead  reckoning  data  that  is  accurate 
in  the  neighborhood  of  a  place.  With  both  maps 
available,  the  robot  can  quickly  plan  a  path  between 
arbitrary  locations,  and  then  define  a  sequence  of 
behaviors  that  will  move  the  robot  along  the  select¬ 
ed  path.  Robust  performance  is  achieved  by  divid¬ 
ing  the  computational  processes  into  two  parallel 
operations.  Time-critical,  low-level  behaviors  like 
driving  and  steering  in  the  exploratory  mode  are 
controlled  by  an  onboard  computer  that  uses  sonar 
data  as  input  to  simple  subsumption-based  algo¬ 
rithms.  Higher  level,  more  computationally  intense, 
and  less-time-critical  activities  like  place  designa¬ 
tion,  map  making,  display  generation,  and  path 
planning  are  performed  in  parallel  on  a  remote 
computer  that  fetches  sonar  data  and  issues 
high-level  commands  via  a  radio  link. 

Teleoperation  to  robotics  at  Langley  Research 
Center 

93A18569 

PENNINGTON,  JACK  E.  (NASA,  Langley  Re¬ 
search  Center,  Hampton,  VA)  Journal  of  Applied 
Intelligence,  vol.  2,  1992,  p.  155-162. 

Experience  of  NASA  Langley  Research  Center  in 
teleoperation,  telerobotics,  and  robotics  in  applica¬ 
tions  related  to  possible  space  tasks  is  reviewed. 
Shared  control  based  on  manual  and  sensor  blend¬ 
ing  in  a  rate  command  control  system  made  it  pos¬ 
sible  to  simultaneously  control  two  or  more  manipu¬ 
lators  executing  a  telerobotic  task  from  a  single 
hand  controller.  It  is  concluded  that  telerobotics 
combines  the  best  features  of  teleoperation  and 
robotics  and  is  sufficiently  mature  for  simple  space 
tasks.  Robotics  is  considered  to  be  feasible  but  less 
mature  and  must  be  highly  reliable. 

Telerobotics,  automation,  and  human  supervisory 
control 

93A17571 

SHERIDAN,  THOMAS  B.  (MIT,  Cambridge,  MA) 
Cambridge,  MA,  MIT  Press,  1992,  414  p. 

A  survey  is  presented  of  teleoperation,  telerobotics, 
and  supervisory  control.  This  is  a  new  form  of 
technology  that  allows  humans  to  work  through 
machines  in  hazardous  environments  and  control 


complex  systems  such  as  aircraft  and  nuclear  power 
plants.  The  general  topics  addressed  are:  theory  and 
models  of  supervisory  control;  supervisory  control 
of  anthropomorphic  teleoperators  for  space,  under¬ 
sea,  and  other  applications;  supervisory  control  in 
transportation,  process,  and  other  automated  sys¬ 
tems;  and  social  implications  of  telerobotics,  auto¬ 
mation,  and  supervisory  control. 

Optimal  motion  planning  of  a  multiple-robot  system 
based  on  decomposition  coordination 

93A17384 

CELA,  ARBEN  S.;  HAMAM,  YSKAUNDAR 
(Ecole  Superieure  d'Ingenieurs  en  Electrotechnique 
et  Electronique,  Noisy-le-Grand,  France)  IEEE 
Transactions  on  Robotics  and  Automation  (ISSN 
1042-296X),  vol.  8,  no.  5,  Oct.  1992,  p.  585-596. 

A  solution  to  the  problem  of  optimal  trajectory 
planning  of  a  multiple-robot  system  in  the  presence 
of  obstacles  is  presented  which  is  based  on  nonlin¬ 
ear  programming  and  decomposition  coordination. 

A  decomposition  method  is  used  to  reduce  the  prob¬ 
lem  to  the  single-robot  level,  and  the  augmented 
Lagrangian  method  is  used  to  treat  the  problem  of  a 
single  robot  in  the  presence  of  obstacles.  When 
applied  to  large-scale  optimization  problems,  the 
methodology  allows  the  possibility  of  a  modular 
approach  to  the  solution.  The  approach,  which  is 
also  applicable  to  the  single-robot  case,  reduces  the 
computational  effort  and  allows  the  use  of  the  best 
optimization  algorithm  for  a  given  optimization 
problem. 

Position  estimation  for  an  autonomous  mobile  robot 
in  an  outdoor  environment 

93A17383 

TALLURI,  RAJ;  AGGARWAL,  J.  K.  (Texas  Univ., 
Austin)  IEEE  Transactions  on  Robotics  and  Auto¬ 
mation  (ISSN  1042-296X),  vol.  8,  no.  5,  Oct.  1992, 
p.  573-584. 

This  paper  presents  a  solution  to  the  position  esti¬ 
mation  problem  of  an  autonomous  land  vehicle 
navigating  in  an  unstructured  mountainous  terrain. 

A  digital  elevation  map  (DEM)  of  the  area  in  which 
the  robot  is  to  navigate  is  assumed  to  be  given.  It  is 
also  assumed  that  the  robot  is  equipped  with  a  cam¬ 
era  that  can  be  panned  and  tilted,  a  compass,  and  an 
altimeter.  No  recognizable  landmarks  are  assumed 
to  be  present  in  the  environment  in  which  the  robot 
is  to  navigate,  and  the  robot  is  not  assumed  to  have 
an  inital  estimate  of  its  position.  The  solution  pre¬ 
sented  here  makes  use  of  the  DEM  information,  and 
structures  the  problem  as  a  constrained  search  para¬ 
digm  by  searching  the  DEM  for  the  possible  robot 
location.  The  algorithm  is  made  robust  to  errors  in 
the  imaging  process  by  accounting  for  worst  case 
errors.  The  approach  is  tested  using  real  terrain  data 
of  areas  in  Colorado  and  Texas.  The  method  is 
suitable  for  use  in  outdoor  mobile  robots  and  plane¬ 
tary  rovers. 
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Exact  robot  navigation  using  artificial  potential 
functions 

93A17380 

RIMON,  ELON;  KODITSCHEK,  DANIEL  E. 
(Stanford  Univ.,  CA);  (Yale  Univ.,  New  Haven, 

CT)  IEEE  Transactions  on  Robotics  and  Automa¬ 
tion  (ISSN  1042-296X),  vol.  8,  no.  5,  Oct.  1992,  p. 
501-518. 

A  new  methodology  for  exact  robot  motion  plan¬ 
ning  and  control  is  presented  which  combines  the 
purely  kinematic  path  planning  problem  with  the 
lower  feedback  controller  design.  It  is  shown,  in 
particular,  how  navigation  functions  can  be  con¬ 
structed  from  a  geometric  description  of  the  free 
configuration  space.  The  computational  complexity 
of  the  resulting  algorithm  is  assessed,  and  results  of 
a  series  of  simulation  studies  are  presented.  Finally 
suggestions  are  made  concerning  the  extension  of 
ideas  presented  here. 

Adaptive  multisensor  fusion  for  planetary 
exploration  rovers 

93A13664 

COLLIN,  MARIE-FRANCE;  KUMAR,  KRISHEN; 
PAMPAGNIN,  LUC-HENRI  (NASA,  Johnson 
Space  Center,  Houston,  TX);  (ITMI,  Meylan, 
France)  In:  Artificial  intelligence,  robotics,  and 
automatic  control,  applied  to  space  /Intelligence 
artificielle,  robotique  et  automatique,  appliquees  a 
1'espace/.  Toulouse,  Cepadues-Editions,  1992,  p. 
113-116.  Research  supported  by  NATO. 

The  purpose  of  the  adaptive  multisensor  fusion 
system  currently  being  designed  at  NASJohnson 
Space  Center  is  to  provide  a  robotic  rover  with 
assured  vision  and  safe  navigation  capabilities  dur¬ 
ing  robotic  missions  on  planetary  surfaces.  Our 
approach  consists  of  using  multispectral  sensing 
devices  ranging  from  visible  to  microwave  wave¬ 
lengths  to  fulfill  the  needs  of  perception  for  space 
robotics.  Based  on  the  illumination  conditions  and 
the  sensors  capabilities  knowledge,  the  designed 
perception  system  should  automatically  select  the 
best  subset  of  sensors  and  their  sensing  modalities 
that  will  allow  the  perception  and  interpretation  of 
the  environment.  Then,  based  on  reflectance  and 
emittance  theoretical  models,  the  sensor  data  are 
fused  to  extract  the  physical  and  geometrical  surface 
properties  of  the  environment  surface  slope,  dielec¬ 
tric  constant,  temperature  and  roughness.  The  theo¬ 
retical  concepts,  the  design  and  first  results  of  the 
multi  sensor  perception  system  are  presented. 

On  a  trajectory  tracking  problem  for  nonlinear 
control  systems 

93A13166 

CHEN,  GUANRONG  (Houston  Univ.,  TX)  In: 

IEEE  Conference  on  Decision  and  Control,  30th, 
Brighton,  United  Kingdom,  Dec.  11-13,  1991,  Pro¬ 
ceedings.  Vol.  3  (A93-13001  02-63).  New  York, 
Institute  of  Electrical  and  Electronics  Engineers, 

Inc.,  1991,  p.  2435-2440.  Research  supported  by 
Univ.  of  Houston. 

An  approach  for  studying  typical  point-to-point 


trajectory  tracking  problems  for  nonlinear  control 
systems  that  possess  a  global  linearization  is  pro¬ 
posed.  The  trajectory  constraints  include  both  the 
inequality  and  the  equality  (interpolatory)  types.  For 
purposes  of  theoretical  analysis,  system-behavior 
understanding,  and  controller  design,  a  minimum 
control-energy  criterion  for  the  linearized  system  is 
used.  Under  this  optimality  criterion,  a  characteriza¬ 
tion  result  for  describing  all  the  possible  solutions 
of  such  trajectory  tracking  problems  is  established. 
Moreover,  the  general  structure  is  found  in  explicit 
closed-form  for  these  solutions.  The  research  is 
motivated  by  a  specific  example  of  robotic  trajecto¬ 
ry  planning.  Some  computer  simulation  graphs  on 
the  robotic  trajectory  planning  problem  are  included. 

Repeatable  generalized  inverse  control  strategies  for 
kinematically  redundant  manipulators 

93  A 1 3 1 65 

ROBERTS,  RODNEY  G.;  MACIEJEWSKI,  AN¬ 
THONY  A.  (Purdue  Univ.,  West  Lafayette,  IN)  In: 
IEEE  Conference  on  Decision  and  Control,  30th, 
Brighton,  United  Kingdom,  Dec.  11-13,  1991,  Pro¬ 
ceedings.  Vol.  3  (A93-1300I  02-63).  New  York, 
Institute  of  Electrical  and  Electronics  Engineers, 

Inc.,  1991,  p.  2428-2434.  Research  supported  by 
NEC  Corp.  and  TRW,  Inc. 

The  issue  of  generating  a  repeatable  control  strategy 
which  possesses  the  desirable  physical  properties  of 
a  particular  generalized  inverse  is  addressed.  This  is 
done  by  first  characterizing  repeatable  strategies 
using  othonormal  basis  functions  to  describe  the 
null  space  of  these  transformations.  The  optimal 
repeatable  inverse  is  then  obtained  by  projecting 
the  null  space  of  the  desired  inverse,  in  an  integral 
norm  sense,  from  the  set  of  all  inverses  spanned 
by  the  selected  basis  functions.  This  technique  is 
illustrated  for  a  planar,  three-degree-of-freedom 
manipulator. 

An  automated  collision-avoidance  motion  planner 
among  moving  objects  or  machines 

93  A 131 

WU,  C.  FI.;  FRETER,  K.;  LEE,  D.  T.;  HWANG,  K. 
S.  (Northwestern  Univ.,  Evanston,  IL)  In:  IEEE 
Conference  on  Decision  and  Control,  30th, 

Brighton,  United  Kingdom,  Dec.  11-13,  1991,  Pro¬ 
ceedings.  Vol.  3  (A93-13001  02-63).  New  York, 
Institute  of  Electrical  and  Electronics  Engineers, 

Inc.,  1991,  p.  2422-2427. 

An  automated  motion  planner  is  proposed  for  coor¬ 
dinating  collision-free  motions  among  dynamic 
moving  objects  or  machines  in  a  3-D  space.  Based 
on  the  concept  of  modified  path-velocity  decompo¬ 
sition,  an  algorithm  is  proposed  for  evaluating  the 
performance  of  different  motion  strategies.  As  a 
result,  the  motion  planner  will  coordinate  collision- 
free  motions  for  multiple  uncontrollable/controllable 
objects  moving  in  a  shared  environment.  A  dynamic 
object  can  also  be  a  multibody  object  such  as  a 
robot.  The  applicability  of  the  proposed  approach  is 
demonstrated  through  two  examples.  The  results 
show  that  multiple  controllable  and  uncontrollable 
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machines  can  work  together  in  an  industrial  envi¬ 
ronment  by  efficiently  avoiding  collisions. 

Collision  avoidance  in  a  multiple-robot  system 
using  intelligent  control  and  neural  networks 

93A13008 

SHIN,  KANG  G.;  CUI,  XIANZHONG  (Michigan 
Univ.,  Ann  Arbor)  In:  IEEE  Conference  on  Deci¬ 
sion  and  Control,  30th,  Brighton,  United  Kingdom, 
Dec.  11-13,  1991,  Proceedings.  Vol.  1  (A93-13001 
02-63).  New  York,  Institute  of  Electrical  and  Elec¬ 
tronics  Engineers,  Inc.,  1991,  p.  130-135. 

A  new  hierarchical  collision  avoidance  scheme  is 
proposed  to  coordinate  multiple  robots  in  a  common 
workspace  by  combining  the  techniques  of  intelli¬ 
gent  control  and  neural  networks  (NNs).  The  high 
level  in  the  hierarchy  is  formed  by  a  knowledge- 
based  coordinator  (KBC)  and  an  NN-based  predic¬ 
tor,  and  the  low  level  consists  of  the  robots  to  be 
coordinated.  The  authors  state  the  problem  of  coor¬ 
dinating  multiple  robots  for  collision  avoidance  and 
the  basic  principles  of  the  KBC.  The  knowledge 
acquisition  and  representation  of  collision  detection 
and  avoidance  for  both  cylindrical-  and  revolute- 
type  robots  are  discussed.  Design  of  the  NN-based 
predictor  and  the  KBC  is  summarized.  The  pro¬ 
posed  scheme  was  tested  extensively  via  simula¬ 
tions  for  both  types  of  robots,  showing  promising 
performance. 

Machine  learning  in  motion  control 

94N70949 

SU,  RENJENG;  KERMICHE,  NOUREDDINE  In 
its  First  Annual  Symposium.  Volume  1:  Plenary 
Session  15  p  (SEE  N94-70944  07-66)  Avail:  CASI 
HC  A03/MF  A04 

The  existing  methodologies  for  robot  programming 
originate  primarily  from  robotic  applications  to 
manufacturing,  where  uncertainties  of  the  robots  and 
their  task  environment  may  be  minimized  by  repeat¬ 
ed  off-line  modeling  and  identification.  In  space 
application  of  robots,  however,  a  higher  degree  of 
automation  is  required  for  robot  programming  be¬ 
cause  of  the  desire  of  minimizing  the  human  inter¬ 
vention.  We  discuss  a  new  paradigm  of  robotic 
programming  which  is  based  on  the  concept  of 
machine  learning.  The  goal  is  to  let  robots  practice 
tasks  by  themselves  and  the  operational  data  are 
used  to  automatically  improve  their  motion  perfor¬ 
mance.  The  underlying  mathematical  problem  is  to 
solve  the  problem  of  dynamical  inverse  by  iterative 
methods.  One  of  the  key  questions  is  how  to  ensure 
the  convergence  of  the  iterative  process.  There  have 
been  a  few  small  steps  taken  into  this  important 
approach  to  robot  programming.  We  give  a  repre¬ 
sentative  result  on  the  convergence  problem. 

Critical  issues  in  robot-human  operations  during  the 
early  phases  of  the  Space  Station  Program 

93N72282 

LAUDERBAUGH,  L.  KEN;  KONDRASKE, 
GEORGE;  WALKER,  MICHAEL  W.;  CHANG, 
KAI-HSIUNG  (Rensselaer  Polytechnic  Inst.,  Troy, 


NY.);  (Texas  Univ.,  Arlington.);  (Michigan  Univ., 
Ann  Arbor.)  Sponsored  by  the  Universities  Space 
Research  Association,  Columbia,  MD,  and  Califor¬ 
nia  Univ.,  San  Diego,  La  Jolla,  CA  Avail:  CASI 
HC  A04/MF  A01 

A  multidisciplinary  team  of  engineers  and  scientists 
from  five  universities  examined  major  aspects  of 
safety  for  the  Flight  Telerobotic  Servicer  (FTS) 
program.  The  team  identified  several  requirements 
for  safe  operation  of  the  FTS.  They  include  an 
overall  monitor  (or  watchdog)  system,  a  single  clear 
line  of  control  at  all  times,  special  modes  and  sen¬ 
sors  that  apply  when  an  astronaut  is  inside  the  FTS' 
work  envelope,  confirmed  reporting  of  current  sta¬ 
tus,  and  manual  modes  for  testing  and  emergency 
situations.  Topics  discussed  include:  robot  safety  - 
industrial  experience;  human  performance  consid¬ 
erations;  NASREM  telerobot  control  system;  soft¬ 
ware  safety  for  the  FTS;  and  designing  for  safety. 

Collision  avoidance  of  mobile  robots  in 
non-stationaiy  environments 

93N71641 

KYRIAKOPOULOS,  K.  J.;  SARIDIS,  G.  N.  Avail: 
CASI  FIC  A03/MF  A01 

A  control  strategy  for  real-time  collision  avoidance 
of  a  mobile  robot  in  an  environment  containing 
moving  obstacles  is  proposed.  A  dynamic  model  of 
the  robot,  the  constraints  and  assumptions  are  pre¬ 
sented.  Objects,  including  the  robot,  are  modelled  as 
convex  polyhedra.  Collision  avoidance  is  guaranteed 
if  the  minimum  distance  between  the  robot  and  the 
objects  is  nonzero.  A  nominal  trajectory  is  assumed 
to  be  known  from  off-line  planning.  The  main  idea 
is  to  change  the  velocity  along  the  nominal  trajecto¬ 
ry  so  that  collisions  are  avoided.  Furthermore,  con¬ 
sistency  with  the  nominal  plan  is  desirable.  The 
process  is  formulated  as  an  optimization  problem 
and  a  close  to  optimal  solution  is  obtained.  Simu¬ 
lation  results  verify  the  value  of  the  proposed 
strategy. 

An  efficient  minimum  distance  and  collision 
estimation  technique  for  on-line  motion  planning 
of  robotic  manipulation 

93N71638 

KYRIAKOPOULOS,  K.  J.;  SARIDIS,  G.  N.  Avail: 
CASI  HC  A03/MF  A01 

In  this  paper,  an  efficient  method  for  computing  the 
minimum  distance  between  objects  is  presented. 
Objects  are  assumed  to  be  convex  polygons.  For 
every  object  a  coordinate  frame  is  assigned.  In  the 
beginning  the  coordinate  frames  coincide;  when  the 
objects  move,  their  new  description  can  be  easily 
obtained  if  it  is  expressed  using  the  relative  position 
and  orientation  of  the  coordinate  frames.  If  instead 
of  using  the  Euclidean  norm,  metric(l)  and  met- 
ric(infinity)  are  used  to  represent  the  distance,  then 
a  linear  programming  problem  is  formulated.  An 
off-line  plan  is  assumed  to  preexist  described  by 
two  functions:  one  describing  the  geometry  of  the 
path  and  the  other  the  motion  in  time  along  this 
trajectory.  In  this  framework,  an  architecture  for 
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on-line  collision  estimation  and  motion  planning  is 
outlined. 

Planning  tasks  with  an  emergent  connectionist/ 

symbolic  system 

93N71631 

MOED,  MICHAEL  C.  Avail:  CASI  HC  A03/ 

MF  A01 

Rule  based  systems  for  planning  abstract  robotic 
tasks  often  suffer  by  making  rule  instantiations 
which  do  not  achieve  the  desired  rule  effects.  In 
many  of  these  instances,  general  rules  fail  because  a 
specific  instantiation  is  an  exception  to  the  general 
case,  and  the  resulting  plan  is  faulty.  In  other  in¬ 
stances,  the  specific  instantiation  fails  to  achieve  the 
desired  effect  due  to  unmodeled  perturbations  in  the 
environmental  state.  To  overcome  these  problems,  a 
probability  value  can  be  associated  with  specific 
instantiations  of  general  rules  which  quantitatively 
describes  the  likelihood  that  the  specific  rule 
achieves  the  stated  effect.  From  a  given  initial  state, 
a  feasible  plan  can  be  developed  that  satisfies  a 
stated  goal  by  sequencing  rules  which  have  highly 
probable  desired  effects.  The  uncertainty  that  the 
plan  achieves  the  stated  goal  can  be  computed  from 
the  probability  of  effect  of  each  rule  used  in  the 
plan.  However,  since  the  number  of  possible 
instantiations  of  general  rules  in  a  rule  store  may  be 
excessive,  all  specific  rules  cannot  be  tested  thor¬ 
oughly  and  maintained  in  memory  with  correspond¬ 
ing  probability  of  effect  values.  Instead,  probability 
of  effect  values  for  untested  instantiations  must 
somehow  be  reliably  extracted  from  specific,  tested 
rules.  The  following  problems  are  discussed:  (1) 
developing  a  methodology  for  finding  sets  of  specif¬ 
ic  rules  which  have  a  high  probability  of  achieving 
a  desired  effect  from  a  base  which  contains  many 
general  rules  and  a  limited  number  of  specific 
instantiations  of  these  general  rules,  and  (2)  se¬ 
quencing  rules  which  have  a  high  probability  of 
effect  to  develop  a  plan  of  abstract  tasks  which 
achieves  a  desired  goal. 

Modeling  and  planning  of  distributed  robot  sensing 

93N71602 

SMITH,  RANDALL  C.;  CHEESEMAN,  PETER 
C.;  NITZAN,  DAVID  Avail:  CASI  HC  A03 
In  the  work  statement  of  Proposal  ECU  81-1 14R  we 
identified  components  of  our  three-year  study  of 
distributed  sensing:  (1)  planning  and 
information-gathering;  (2)  sensor  planning;  and  (3) 
experimental  verification.  We  have  made  substantial 
progress  on  the  components  of  the  statement  of 
work,  with  most  emphasis  during  the  first  half  of 
the  project  centered  on  component  (1),  which  we 
perceive  as  the  most  difficult.  This  effort  has  two 
general  results:  (1)  development  of  an  analytic 
method  useful  in  judging  the  necessity  and  applica¬ 
bility  of  a  sensing  step  in  providing  information  for 
a  particular  goal;  and  (2)  development  of  a 
general-purpose  planner  which  will  utilize  this  anal¬ 
ysis  tool  to  select  among  available  sensors  and 
sensing  strategies  in  a  given  task.  These  develop¬ 


ments  of  component  (1)  are  reviewed.  Component 
(2),  which  includes  modeling  specific  sensor  types 
and  utilizing  expert  knowledge  in  sensor  selection, 
was  scheduled  to  receive  emphasis  in  the  second 
and  third  years  of  the  project,  and  our  work  on  it 
has  begun.  Component  (3)  will  be  addressed  at  the 
conclusion  of  the  project  in  the  third  year. 

The  Planning  Coordinator  (PCOORD)  for  robust 
error  recovery  and  dynamic  on-line  planning  of 
robotic  tasks 

93N71241 

FARAH,  J.  J.;  KELLEY,  ROBERT  B.  Avail: 

CASI  HC  A03/MF  A01 

The  Planning  Coordinator  is  a  logical  extension  to 
the  Coordination  Level  of  the  Intelligent  Machine 
Model,  functioning  to  provide  a  heretofore  unavail¬ 
able  platform  for  robust  error  recovery  and  dynamic 
on-line  planning  by  autonomous  and  semi-autono¬ 
mous  robotic  systems.  This  paper  introduces  the 
Planning  Coordinator  and  focuses  upon  its  macro¬ 
structure,  interfaces,  and  functional  description,  with 
respect  to  its  role  as  the  mechanism  whereby  an 
existing  robotic  system  requiring  significant  human 
intervention  can  be  made  more  autonomous,  thus 
becoming  more  robust. 

Development  of  a  control  system  for  a  pair  of 
robotic  platforms 

93N71189 

COSENTINO,  JAMES  L.  Avail:  CASI  HC 
A05/MF  A01 

This  thesis  is  a  discussion  of  the  development  of  a 
control  system  for  a  pair  of  three-degree-of-freedom 
robotic  platforms.  The  platform  system  and  its  com¬ 
puter  support  are  described  at  both  the  hardware 
and  the  software  levels,  including  a  section  detailing 
diagnostics  which  can  be  run  in  the  event  of  specif¬ 
ic  system  errors.  The  design  and  implementation  of 
a  PID  controller  for  the  platforms  based  on 
experimentally-determined  system  dynamics  is  de¬ 
scribed.  Finally,  the  tracking  performance  of  each 
joint  controller  is  examined.  A  path  planner  is  used 
to  map  a  smooth  trajectory  between  starting  and 
destination  positions,  and  the  controller's  tracking 
ability  is  observed  along  this  path. 

Robot  planning,  execution,  and  monitoring  in  an 
uncertain  environment 

93N71181 

MUNSON,  JOHN  H.  Presented  at  the  Second 
IJCAI,  London,  England,  1-3  Sep.  1971  Sponsored 
in  part  by  ARPA  Avail:  CASI  HC  A03/MF  A01 
An  intelligent  robot,  operating  in  an  external  envi¬ 
ronment  that  cannot  be  fully  modeled  in  the  robot's 
software,  must  be  able  to  monitor  the  success  of  its 
execution  of  a  previously  generated  plan.  This  paper 
outlines  a  unified  formalism  for  describing  and 
relating  the  various  functions  of  a  robot  operating  in 
such  an  environment.  After  exploring  the  distinction 
between  the  external  world  and  the  robot's  internal 
model  of  it,  and  the  distinction  between  actions  that 
interact  with  the  world  and  the  robot's  descriptions 
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of  those  actions,  we  formalize  the  concepts  of  a 
plan  and  of  its  execution.  Current  developments  at 
Stanford  Research  Institute,  and  the  benchmark  idea 
of  an  'ultimate'  rational  robot,  are  both  analyzed  in 
this  framework. 

Control  of  a  Cartesian  robot 

93N70978 

FRANKLIN,  JUDY;  NOYES,  TERRI;  POCOCK, 
GERRY  In  Inst,  for  Applied  FORTH  Research, 

Inc.,  The  Journal  of  FORTH  Application  and  Re¬ 
search,  Volume  1,  1983  p  23-31  (SEE  N93-70975 
04-61)  Avail:  Issuing  Activity 
The  work  in  control  applications  at  the  Laboratory 
for  Perceptual  Robotics  has  been  directed  toward  a 
prototype  Cartesian  Assembler  donated  to  the  labo¬ 
ratory  by  General  Electric  of  Schenectady,  New 
York.  The  machine  and  some  of  the  hardware  inter¬ 
faces  are  described  along  with  the  low  level  control¬ 
ling  schemes  for  point-to-point  position/velocity 
control.  An  emulation  of  single  axis  controllers  is 
shown  to  be  an  effective  control  method.  Encod¬ 
er/positional  information  is  the  basis  of  this  low 
level  control  structure  which  will  later  be  tailored 
for  use  with  processors  devoted  to  each  axis.  High 
level  control  issues  such  as  adaptive  learning  tech¬ 
niques  are  addressed. 

Machine  learning  for  flexible  robotics 

93N70070 

DEJONG,  GERALD  F.  Avail:  CASI  HC  A02/ 

MF  A01 

Robotic  planning,  if  it  is  to  be  successful  in  real 
world  situations,  must  find  some  way  to  side-step 
the  now  well-documented  obstacles  to  classical  AI 
planning.  These  recent  results  show  that  the  compu¬ 
tational  complexity  of  standard  planning  is  unac¬ 
ceptable  even  with  drastic  and  untenable  simplifying 
assumptions  about  the  world.  The  source  of  com¬ 
plexity  in  real  world  robotic  domains  includes  the 
problems  of  data  uncertainty,  large  amounts  of  data 
to  consider,  as  well  as  the  problem  of  tractably 
producing  plans  according  to  the  given  domain 
rules.  Pretending  that  these  complexities  do  not 
exist  relegates  a  computer  system  to  a  trivialized 
micro-world  with  little  hope  of  applications  to  the 
real  world.  The  research  of  this  grant  has  been 
directed  towards  dealing  with  the  real  world  con¬ 
straints  that  artificial  intelligence  robotics  systems 
must  address.  We  have  made  significant  progress  on 
two  fronts.  The  first  investigates  an  integrated  ap¬ 
proach  to  planning  wherein  a  classical  a  priori  plan¬ 
ner  is  augmented  with  reactive  abilities.  The  second 
thrust  of  this  grant  explores  a  new  approach  called 
permissive  planning.  We  have  implemented  our 
ideas  in  the  GRASPER  system  which  has  capabili¬ 
ties  to  monitor  execution  of  its  plans  and  to  tune  its 
model  of  the  world  on  failure  through  use  of  explic¬ 
it  approximations. 

Experience  with  a  task  control  architecture  for 
mobile  robots 

93N70023 


LIN,  LONG-JI;  SIMMONS,  REID;  FEDOR, 
CHRISTOPHER  Avail:  CASI  HC  A03/MF  A01 
This  paper  presents  a  general-purpose  architecture 
for  controlling  mobile  robots,  and  describes  a  work¬ 
ing  mobile  manipulator  which  uses  the  architecture 
to  operate  in  a  dynamic  and  uncertain  environment. 
The  target  of  this  work  is  to  develop  a  distributed 
robot  architecture  for  planning,  execution,  monitor¬ 
ing,  exception  handling,  and  multiple  task  coordina¬ 
tion.  We  report  our  progress  to  date  on  the  architec¬ 
ture  development  and  the  performance  of  the  work¬ 
ing  robot.  In  particular,  we  discuss  temporal  reason¬ 
ing,  execution  monitoring,  and  context-dependent 
exception  handling. 

Teleoperation  with  virtual  force  feedback 

94N20293 

ANDERSON,  R.  J.  Presented  at  the  Society  of 
Photo-Optical  Instrumentation  Engineer's  (SPIE) 
International  Symposium  on  Optical  Tools  for  Man¬ 
ufacturing  and  Advanced  Automation,  Boston,  MA, 
7-10  Sep.  1993  Avail:  CASI  HC  A02/MF  A01 
In  this  paper  we  describe  an  algorithm  for  generat¬ 
ing  virtual  forces  in  a  bilateral  teleoperator  system. 
The  virtual  forces  are  generated  from  a  world  model 
and  are  used  to  provide  real-time  obstacle  avoidance 
and  guidance  capabilities.  The  algorithm  requires 
that  the  slaves  tool  and  every  object  in  the  environ¬ 
ment  be  decomposed  into  convex  polyhedral  Primi¬ 
tives.  Intrusion  distance  and  extraction  vectors  are 
then  derived  at  every  time  step  by  applying  Gilbert's 
polyhedra  distance  algorithm,  which  has  been  adapt¬ 
ed  for  the  task.  This  information  is  then  used  to 
determine  the  compression  and  location  of  nonlinear 
virtual  spring-dampers  whose  total  force  is  summed 
and  applied  to  the  manipulator/teleoperator  system. 
Experimental  results  validate  the  whole  approach, 
showing  that  it  is  possible  to  compute  the  algorithm 
and  generate  realistic,  useful  pseudo  forces  for  a 
bilateral  teleoperator  system  using  standard  VME 
bus  hardware. 

Self-organization  via  active  exploration  in  robotic 
applications.  Phase  2:  Hybrid  hardware  prototype 

94N20212 

OEGMEN,  HALUK  Sponsored  by  NASA.  Lyndon 
B.  Johnson  Space  Center  Avail:  CASI  HC  A04/ 

MF  A01 

In  many  environments  human-like  intelligent  behav¬ 
ior  is  required  from  robots  to  assist  and/or  replace 
human  operators.  The  purpose  of  these  robots  is  to 
reduce  human  time  and  effort  in  various  tasks.  Thus 
the  robot  should  be  robust  and  as  autonomous  as 
possible  in  order  to  eliminate  or  to  keep  to  a  strict 
minimum  its  maintenance  and  external  control. 

Such  requirements  lead  to  the  following  properties: 
fault  tolerance,  self  organization,  and  intelligence.  A 
good  insight  into  implementing  these  properties  in  a 
robot  can  be  gained  by  considering  human  behavior. 
In  the  first  phase  of  this  project,  a  neural  network 
architecture  was  developed  that  captures  some  fun¬ 
damental  aspects  of  human  categorization,  habit, 
novelty,  and  reinforcement  behavior.  The  model. 
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called  FRONTAL,  is  a  'cognitive  unit'  regulating  the 
exploratory  behavior  of  the  robot.  In  the  second 
phase  of  the  project,  FRONTAL  was  interfaced 
with  an  off-the-shelf  robotic  arm  and  a  real-time 
vision  system.  The  components  of  this  robotic  sys¬ 
tem,  a  review  of  FRONTAL,  and  simulation  studies 
are  presented  in  this  report. 

Robot  skill  learning  and  the  effects  of  basis  function 

choice 

94N20135 

SCHNEIDER,  J.  G.;  BROWN,  C.  M.  Avail:  CAS1 
HC  A03/MF  A01 

We  present  a  computational,  constructive  theory  of 
tunable,  open  loop  trajectory  skills.  A  skill  is  a 
controller  whose  outputs  achieve  any  task  in  a  space 
characterized  by  n  parameters,  n  greater  than  1. 
Throwing  a  ball  at  a  target  is  a  3-dimensional  task 
if  the  target  may  be  anywhere  within  a  3-dimen¬ 
sional  volume.  Repetitious  pick  and  place  tasks  are 
zero-dimensional,  and  thus  not  skills.  Skills  are 
performed  open  loop  for  speed  reasons:  we  assume 
the  entire  command  sequence  is  generated  before 
any  feedback  can  become  available.  We  do  not 
assume  prior  knowledge  of  plant  or  task  models,  so 
skills  must  be  at  least  partly  learned.  A  skill  output 
is  a  vector  of  values  —  in  our  work  so  far  it  is  gen¬ 
erated  as  the  sum  of  a  base  vector  and  a  weighted 
change  vector  whose  weight  accomplishes  the  tun¬ 
ing.  Learning  consists  of  a  search  for  the  best  set  of 
base  and  change  vectors.  An  interpretation  process 
maps  skill  outputs  into  sequences  of  commands  for 
the  plant  by  using  basis  functions  (given  a  priori  in 
this  paper).  The  basis  functions  may  be  arbitrarily 
complex.  We  claim  that  appropriate  basis  functions 
can  speed  up  the  learning  process  and  overcome  the 
limitations  of  the  linear  trajectory  tuning  algorithm. 
This  report  describes  a  skill  learning  algorithm  and 
experiments  done  with  various  basis  functions  and 
control  methods  for  a  one-dimensional  throwing 
task.  It  concludes  with  a  discussion  of  future  work 
in  learning  basis  functions,  higher  dimensional 
tasks,  and  comparisons  against  common  learning 
and  control  algorithms. 

Robot  navigation  in  unknown  terrains:  Introductory 
survey  of  non-heuristic  algorithms 

94N 19300 

RAO,  N.  S.  V.;  KARETI,  S.;  SHI,  WEIMIN; 
IYENGAR,  S.  S.  (Old  Dominion  Univ.,  Norfolk, 
VA.);  (Old  Dominion  Univ.,  Norfolk,  VA.);  (Louisi¬ 
ana  State  Univ.,  Baton  Rouge.)  Avail:  CASI  HC 
A04/MF  A01 

A  formal  framework  for  navigating  a  robot  in  a 
geometric  terrain  through  an  unknown  set  of  obsta¬ 
cles  is  considered.  Here  the  terrain  model  is  not 
known  a  priori,  but  the  robot  is  equipped  with  a 
sensor  system  (vision  or  touch)  employed  for  the 
purpose  of  navigation.  The  focus  is  restricted  to  the 
non-heuristic  algorithms  which  can  be  theoretically 
shown  to  be  correct  within  a  given  framework  of 
models  for  the  robot,  terrain,  and  sensor  system. 
These  formulations,  although  abstract  and  simplified 


compared  to  real-life  scenarios,  provide  foundations 
for  practical  systems  by  highlighting  the  underlying 
critical  issues.  First,  the  authors  consider  the  algo¬ 
rithms  that  are  shown  to  navigate  correctly  without 
much  consideration  given  to  the  performance  pa¬ 
rameters,  such  as  distance  traversed.  Second,  they 
consider  non-heuristic  algorithms  that  guarantee 
bounds  on  the  distance  traversed  or  the  ratio  of  the 
distance  traversed  to  the  shortest  path  length  (com¬ 
puted  if  the  terrain  model  is  known).  Then  they 
consider  the  navigation  of  robots  with  very  limited 
computational  capabilities  such  as  finite  automata. 

Adaptive  path  planning:  Algorithm  and  analysis 

94N 18926 

CHEN,  PANG  C.  Presented  at  the  6th  International 
Symposium  of  Robotics  Research,  Pittsburgh,  PA,  2 
Oct.  1993  Avail:  CASI  HC  A02/MF  A01 
Path  planning  has  to  be  fast  to  support  real-time 
robot  programming.  Unfortunately,  current  planning 
techniques  are  still  too  slow  to  be  effective,  as  they 
often  require  several  minutes,  if  not  hours  of  com¬ 
putation.  To  alleviate  this  problem,  we  present  a 
learning  algorithm  that  uses  past  experience  to  en¬ 
hance  future  performance.  The  algorithm  relies  on 
an  existing  path  planner  to  provide  solutions  to 
difficult  tasks.  From  these  solutions,  an  evolving 
sparse  network  of  useful  subgoals  is  learned  to 
support  faster  planning.  The  algorithm  is  suitable 
for  both  stationary  and  incrementally-changing  envi¬ 
ronments.  To  analyze  our  algorithm,  we  use  a  previ¬ 
ously  developed  stochastic  model  that  quantifies 
experience  utility.  Using  this  model,  we  characterize 
the  situations  in  which  the  adaptive  planner  is  use¬ 
ful,  and  provide  quantitative  bounds  to  predict  its 
behavior.  The  results  are  demonstrated  with  prob¬ 
lems  in  manipulator  planning.  Our  algorithm  and 
analysis  are  sufficiently  general  that  they  may  also 
be  applied  to  task  planning  or  other  planning  do¬ 
mains  in  which  experience  is  useful. 

Survey  of  collision  avoidance  and  ranging  sensors 
for  mobile  robots,  revision  1 

94N 18474 

EVERETT,  H.  R.;  DEMUTH,  D.  E.;  STITZ,  E.  H. 
Avail:  CASI  HC  A08/MF  A02 
The  past  few  years  have  brought  about  a  tremen¬ 
dous  rise  in  the  envisioned  potential  of  robotic  sys¬ 
tems  and  a  significant  increase  in  the  number  of 
proposed  applications.  In  the  nonindustrial  arena, 
numerous  programs  have  evolved,  each  intending  to 
harness  some  of  this  promise  in  hopes  of  solving 
some  particular  application  need.  Many  of  these 
efforts  are  government  sponsored,  aimed  at  the 
development  of  systems  for  fighting  fires,  handling 
ammunition,  transporting  materials,  conducting 
underwater  search  and  inspection  operations,  and 
patrolling  warehouses  and  storage  areas,  etc.  Many 
of  the  resulting  prototypes,  which  were  initially 
perceived  as  logical  extensions  of  the  traditional 
industrial  robotic  scenarios,  have  met  with  unex¬ 
pected  difficulty  due  to  an  insufficient  supporting 
technology  base.  This  document  provides  some 
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basic  background  on  the  various  noncontact  distance 
measurement  techniques  available,  with  related 
discussion  of  implementation  in  the  acoustical, 
optical,  and  electromagnetic  portions  of  the  energy 
spectrum.  An  overview  of  candidate  systems,  both 
commercially  available  and  under  development,  is 
provided,  followed  by  a  brief  summary  of  research 
currently  underway  in  support  of  the  collision 
avoidance  and  noncontact  ranging  needs  of  a  mobile 
robot. 

Flexible  integration  of  path-planning  capabilities 

94N 18463 

STOBIE,  IAIN  C.;  TAMBE,  MILIND; 
ROSENBLOOM,  PAUL  S.  Avail:  Issuing  Activity 
(Defense  Technical  Information  Center  (DTIC)) 
Robots  pursuing  complex  goals  must  plan  paths 
according  to  several  criteria  of  quality  including 
shortness,  safety,  speed,  and  planning  time.  Many 
sources  and  kinds  of  knowledge,  such  as  maps, 
procedures  and  perception,  may  be  available  or 
required.  Both  the  quality  criteria  and  sources  of 
knowledge  may  vary  widely  over  time,  and  in  gen¬ 
eral  they  will  interact.  One  approach  to  address  this 
problem  is  to  express  all  criteria  and  goals  numeri¬ 
cally  in  a  single  weighted  graph,  and  then  to  search 
this  graph  to  determine  a  path.  Since  this  is  prob¬ 
lematic  with  symbolic  or  uncertain  data  and  inter¬ 
acting  criteria,  we  propose  that  what  is  needed  in¬ 
stead  is  an  integration  of  many  kinds  of  planning 
capabilities.  We  describe  a  hybrid  approach  to  inte¬ 
gration,  based  on  experiments  with  building  simulat¬ 
ed  mobile  robots  using  Soar,  an  integrated 
problem-solving  and  learning  system.  For  flexibility, 
we  have  implemented  a  combination  of  internal 
planning,  reactive  capabilities,  and  specialized  tools. 
We  illustrate  how  these  components  can  comple¬ 
ment  each  other's  limitations  and  produce  plans 
which  integrate  geometric  and  task  knowledge. 

Autonomous  neural  network  controllers  for  adaptive 
material  handling 

94N18331 

KOTTAS,  JAMES;  KUPERSTEIN,  MICHAEL 
Avail:  CASI  HC  A03/MF  A01 
For  robots  to  be  more  useful  in  flexible  manufactur¬ 
ing  and  service  applications,  the  controllers  must  be 
able  to  handle  more  variable  environments.  On  at 
least  two  levels,  conventional  methods  in  robot 
control  have  problems  dealing  with  high  variability. 
At  the  current  level,  conventional  dynamic  control 
formulations  cannot  deal  effectively  with  the  highly 
variable  dynamic  inertial  interactions  between 
nrultijointed  robots  and  payloads.  At  the  task  level, 
the  initial  and  final  positions  for  materials  to  be 
moved  may  change  slightly  but  unexpectedly.  We 
have  developed  autonomous  neural  network  con¬ 
trollers  that  learn  from  their  own  experience  to  deal 
with  environmental  variability  at  these  levels. 

Magnetostrictive  actuators  for  human  sensory 

feedback 

94N 17841 


BRIMHALL,  OWEN  D.;  KNOWLTON,  DANIEL; 
CURTIN,  HOWARD  R.  Avail:  CASI  HC  A03/ 

MF  A01 

The  development  of  actuators  with  enhanced  capa¬ 
bilities  is  critical  to  the  achievement  of  sensory 
feedback  systems  for  intuitive,  real  time  human 
operation  of  telerobotic  systems.  The  objective  of 
this  research  project  was  to  demonstrate  feasibility 
of  new  actuators  using  active  materials  which  will 
enhance  the  capabilities  of  dexterous,  exoskeletal 
feedback  systems  for  telerobotic  applications.  In 
phase  1,  feasibility  of  several  novel  Terfenol-D 
magnetostrictive  actuators  were  demonstrated.  Sev¬ 
eral  prototype  actuators  were  fabricated  and  tested, 
including  resistive  single  axis  brakes,  resistive 
multiaxis  (ball)  brake  joints,  and  linear  motors.  The 
new  actuators  are  relatively  efficient,  responsive, 
small  and  exert  high  forces.  The  actuators  provide 
proportional  forces  and  are  easily  interfaced  with 
digital  electronics.  Several  of  the  actuators  were 
integrated  into  a  simple  demonstration  board.  Phase 
2  will  pursue  advanced  development  of  proportional 
force  resistive  brake  actuators  and  active  linear 
motion  actuators. 

Integrating  reactive  and  deliberative  planning  for 
agents 

94N 17201 

BLYTHE,  JIM;  REILLY,  W.  S.  Avail:  Issuing 
Activity  (Defense  Technical  Information  Center 
(DTIC)) 

Autonomous  agents  that  respond  intelligently  in 
dynamic,  complex  environments  need  to  be  both 
reactive  and  deliberative.  Reactive  systems  have 
traditionally  fared  better  than  deliberative  planners 
in  such  environments,  but  are  often  hard  to  code 
and  inflexible.  To  fill  in  some  of  these  gaps,  we 
propose  a  hybrid  system  that  exploits  the  strengths 
of  both  reactive  and  deliberative  systems.  We  dem¬ 
onstrate  how  out  system  controls  a  simulated  house¬ 
hold  robot  and  compare  our  system  to  a  purely 
reactive  one  in  this  domain.  We  also  look  at  a  num¬ 
ber  of  relevant  issues  in  anytime  planning. 

An  object-oriented  program  specification  for  a 
mobile  robot  motion  control  language 

94N16986 

GRIM,  CARL  J.  Avail:  CASI  HC  A06/MF  A02 
The  Yamabico  Research  Group  at  the  Naval  Post¬ 
graduate  School  is  actively  pursuing  improvements 
in  design  and  implementation  of  applications  for  it's 
family  of  autonomous  mobile  robots.  This  paper 
describes  a  new  high  level  language  for  controlling 
the  Yamabico-11,  sumamed  OOPS-MML 
(Object-Oriented  Program  Specification  for  a  Mo¬ 
bile  robot  Motion  control  Language).  Conceptual 
goals  included  a  user  friendly,  high  level  interface 
coupled  with  a  very  abstract,  efficient  and  compart¬ 
mentalized  architecture  to  employ  a  path  planning 
and  tracking  application  developed  at  NPS.  The 
result  is  a  robust  and  flexible  robot  control  system 
that  is  intended  to  be  implemented  and  employed 
onboard  the  Yamabico-11. 
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Contextually  dependent  control  strategies  for 
manipulation 

94N 16580 

POOK,  POLLY  K.;  BALLARD,  DANA  H.  Avail: 
CAS1  HC  A03/MF  A01 

Traditional  analytic  robotics  defines  grasping  by 
knowing  the  task  geometry  and  the  forces  acting  on 
the  manipulator  precisely.  This  method  is  particular¬ 
ly  important  for  non-compliant  manipulators  with 
few  degrees  of  freedom,  such  as  a  parallel  jaw 
gripper,  that  overconstrain  the  solution  space.  In 
contrast,  the  advent  of  anthropomorphic,  high 
degree-of-freedom  grippers  allows  us  to  use 
closed-loop  strategies  that  depend  heavily  on  the 
task  context  but  do  not  require  precise  positioning 
knowledge.  To  demonstrate,  a  robotic  hand  flips  a 
plastic  egg,  using  the  finger  joint  tendon  tensions  as 
the  the  sole  control  signal.  The  manipulator  is  a 
compliant,  sixteen  degree-of-freedom,  Utah/MIT 
hand  mounted  on  a  Puma  760  arm.  The  completion 
of  each  subtask,  such  as  picking  up  the  spatula, 
finding  the  pan,  and  sliding  the  spatula  under  the 
egg,  is  detected  by  sensing  when  the  tensions  of  the 
hand  tendons  pass  a  threshold.  Beyond  this  use  of 
tendon  tensions  and  the  approximate  starting  posi¬ 
tion  of  the  spatula  and  pan,  no  model  of  the  task  is 
constructed.  The  routine  is  found  to  be  robust  to 
different  spatulas  and  to  changes  in  the  location  and 
orientation  of  the  spatula,  egg,  and  table,  with  some 
exceptions.  The  egg-flipping  example  relies  on 
interpreting  fluctuating  tension  values  within  a 
known  temporal  sequence  of  actions.  For  instance, 
knowing  when  the  manipulator  is  trying  to  touch 
the  pan  with  the  spatula  provides  the  context  to 
interpret  changes  in  tendon  tensions.  Given  the 
success  of  this  task,  we  go  on  to  propose  a  method 
for  analyzing  the  temporal  sensory  output  for  tasks 
that  have  not  been  previously  segmented.  This 
method  suggests  a  means  for  automatically  generat¬ 
ing  robust  force  control  programs  to  perform  previ¬ 
ously  teleoperated  manipulation  tasks. 

The  ESA  telescience  program 
94N 16073 

NAJA,  G.  In  National  Research  Council  Canada, 
Proceedings  of  the  Second  Workshop  on 
Microgravity  Experimentation  7  p  (SEE  N94- 16071 
03-29)  Avail:  Issuing  Activity  (National  Research 
Council,  Publication  Sales  and  Distribution,  Montre¬ 
al  Road,  Ottawa,  Ontario,  K1A  OR6  Canada) 

In  the  Columbus  space  station  Attached  Laboratory, 
there  will  be  40  experiment  racks  but  less  than  two 
crew  members  available  to  tend  them,  meaning  that 
experiment  operations  cannot  be  permanently  per¬ 
formed  by  the  crew.  The  Columbus  Free  Flyer  will 
be  a  completely  automated  laboratory,  serviced 
periodically  by  astronauts.  To  facilitate  the  running 
of  experiments  in  the  Columbus  facilities,  the 
telescience  concept  will  be  used  extensively. 
Telescience  provides  ground-based  users  with  inter¬ 
active  and  transparent  access  to  their  on-orbit  ex¬ 
periments.  Data  such  as  telemetry  and  video  are 
received  by  the  user,  who  then  can  react  by  sending 


telecommands  to  activate  mechanisms  or  robotic 
systems  to  operate  the  experiment.  To  validate  the 
telescience  concept,  to  promote  it,  and  to  train 
telescience  users,  a  testbed  has  been  developed  and 
installed.  The  testbed  has  integrated  five  pilot  ex¬ 
periments  operated  through  a  user  workstation.  The 
testbed  has  also  been  used  in  an  operational  simula¬ 
tion  of  long-duration  space  missions.  A  flight  test  of 
a  telescience  experiment  investigating  Marangoni 
convection  was  performed  in  Nov.  1989  onboard  a 
sounding  rocket  with  complete  success. 

Hidden  Markov  model  approach  to  skill  learning 
and  its  application  to  telerobotics 

94N15502 

YANG,  JIE;  XU,  YANGSHENG;  CHEN,  C.  S. 
Avail:  CASI  HC  A03/MF  A01 
In  this  paper,  we  discuss  the  problem  of  how  hu¬ 
man  skill  can  be  represented  as  a  parametric  model 
using  a  hidden  Markov  model  (HMM),  and  how  a 
HMM-based  skill  model  can  be  used  to  learn  hu¬ 
man  skill.  HMM  is  feasible  to  characterize  two 
stochastic  processes— measurable  action  and  immea¬ 
surable  mental  states— which  are  involved  in  the 
skill  learning.  We  formulated  the  learning  problem 
as  a  multi-dimensional  HMM  and  developed  a  pro¬ 
gramming  system  which  serves  as  a  skill  learning 
testbed  for  a  variety  of  applications.  Based  on  'the 
most  likely  performance'  criterion,  we  can  select  the 
best  action  sequence  from  all  previously  measured 
action  data  by  modeling  the  skill  as  PIMM.  This 
selection  process  can  be  updated  in  real-time  by 
feeding  new  action  data  and  modifying  HMM  pa¬ 
rameters.  We  address  the  implementation  of  the 
proposed  method  in  a  teleoperation-controlled  space 
robot.  An  operator  specifies  the  control  command 
by  a  hand  controller  for  the  task  of  exchanging 
Orbit  Replaceable  Unit,  and  the  robot  learns  the 
operation  skill  by  selecting  the  sequence  which 
represents  the  most  likely  performance  of  the  opera¬ 
tor.  The  skill  is  learned  in  Cartesian  space,  joint 
space,  and  velocity  domain.  The  experimental  re¬ 
sults  demonstrate  the  feasibility  of  the  proposed 
method  in  learning  human  skill  and  teleoperation 
control.  The  learning  is  significant  in  eliminating 
sluggish  motion  and  correcting  the  motion  com¬ 
mand  which  the  operator  mistakenly  generates. 

Robot  programming,  natural  computation  and 
conceptual  graphs 

94N13539 

KAPITANOVSKY,  ALEX  Avail:  Issuing  Activity 
(Tel-Aviv  Univ.,  Exact  Sciences  Library,  Ramat 
Aviv  69978,  Israel) 

A  method  is  developed  for  robot  program  synthesis. 
Currently,  the  programming  of  a  robot  task  is  one 
of  the  major  hurdles  of  robot  application.  The  auto¬ 
mation  of  robot  program  synthesis  will  ease  indus¬ 
trial  application  of  robots.  In  order  to  develop  a 
support  system  for  robot  programming,  it  is  pro¬ 
posed  to  consider  the  natural  information  processing 
by  humans  during  synthesis  and  interpretation  of 
robotic  programs  and  then  to  construct  an  approxi- 


B-27 


mate  conceptual  model  of  the  dynamically  changing 
real  world  which  is  involved.  Such  a  model  must  be 
suitable  (i.e.,  representable  and  executable)  for  effi¬ 
cient  computer  processing.  This  approach  enables 
the  proposed  system  to  provide  a  means  for  auto¬ 
mated  (knowledge  guided)  conversion  of  a  user's 
request,  expressed  in  a  natural  language,  to  the 
appropriate  conceptual  model  of  the  required  task. 
This  model  incorporates  the  information  necessary 
for  understanding,  planning,  and  sensory-guided 
performance  of  the  required  robotic  task.  In  Part  A, 
the  overall  framework  of  the  system  is  defined, 
together  with  the  first  phase  of  an  assembly  pro¬ 
gram  synthesis:  request  specification,  planning  of 
the  valid  assembly  sequences  and  determination  of 
the  required  sources.  Part  B  concerns  the  final 
phase  of  a  robot  program  synthesis  and  its  interpre¬ 
tation  in  changing  conditions.  A  case  study  is  pre¬ 
sented,  to  illustrate  this  approach  on  the  large  fami¬ 
ly  of  multi-axisymmetric  components. 

Application  of  intelligently  adaptable  structure  and 
robot  to  large  space  structure 

94N13277 

SENDA,  KEI;  MUROTSU,  YOSHISADA  In 
NASDA,  The  Second  Workshop  on  Deployment 
and  Assembly  Experiment  of  Large  Space  Structure 
on  Orbit  p  47-53  (SEE  N94-13268  02-12)  Avail: 
CASI  HC  A02/MF  A01 

The  status  of  the  research  on  applying  intelligently 
a  adaptable  structure  and  robot  to  a  large  space 
structure  is  presented.  The  following  items  are  de¬ 
scribed:  (1)  the  selected  research  subjects;  (2)  the 
present  status  of  the  research  on  FFR  (Free  Flying 
Robot)  including  formulation  concerning  FFR,  iner¬ 
tial  parameter  identification  and  manipulator  orbit 
plan  for  controlling  the  attitude  of  rigid  FFR,  study 
on  controlling  of  FFR  with  flexible  manipulator 
applying  virtual  rigid  manipulator  concept,  hardware 
experiments  using  a  model  with  one  or  two  of  two 
link  solid  manipulator,  and  the  one  with  a  two  link 
flexible  manipulator;  (3)  the  intelligently  adaptable 
structure  including  actuator  layout;  (4)  optimum 
attitude  utilization  as  a  docking  mechanism;  and  (5) 
control  of  flexible  space  structure. 

Experiments  in  mobile  robot  navigation  and  range 
imaging 

94N12598 

JONES,  J.  P.;  DORUM,  O.  H.;  ANDERSEN,  C.  S.; 
JACOBSEN,  S.  V.;  JENSEN,  M.  S.;  KIERKEBY, 
N.  O.  S.;  KRISTENSEN,  S.  (Norges  Tekniske 
Hoegskole,  Trondheim.);  (Aalborg  Univ.,  Den¬ 
mark.);  (Aalborg  Univ.,  Denmark.);  (Aalborg  Univ., 
Denmark.);  (Aalborg  Univ.,  Denmark.);  (Aalborg 
Univ.,  Denmark.)  Presented  at  the  8th  Scandanavian 
Conference  on  Image  Analysis,  Tromso,  Norway, 
25-28  May  1993  Avail:  CASI  IIC  A03/MF  A01 
This  paper  describes  some  experiments  in 
sensor-based  mobile  robot  navigation  conducted  at 
Oak  Ridge  National  Laboratory  over  the  past  sev¬ 
eral  years.  Two  implemented  systems  are  described. 
One  uses  a  laser  range  camera  as  the  sole  sensor. 


The  other  uses,  in  addition,  an  array  of  sonars  and  a 
stereo  vision  system.  We  discuss  a  communication 
system  for  heterogeneous  LAN-connected 
multiprocessor  systems,  useful  in  reasonably  large 
development  projects  such  as  these.  We  also  de¬ 
scribe  some  work  on  the  estimation  of  curvature  in 
range  images  which  introduces  a  new  variational 
principle  motivated  by  minimization  of  the  change 
of  curvature.  Application  of  this  principle  is  shown 
to  produce  results  which  are  more  desirable  in  some 
respects  than  those  obtained  using  standard  quadrat¬ 
ic  variation. 

The  rational  behavior  model:  A  multi-paradigm, 
tri-level  software  architecture  for  the  control  of 
autonomous  vehicles 
94N 11874 

BYRNES,  RONALD  B„  JR.  Avail:  CASI  HC 
A14/MF  A03 

There  is  currently  a  very  strong  interest  among 
researchers  in  the  fields  of  artificial  intelligence  and 
robotics  in  finding  more  effective  means  of  linking 
high  level  symbolic  computations  relating  to  mis¬ 
sion  planning  and  control  for  autonomous  vehicles 
to  low  level  vehicle  control  software.  The  diversity 
exhibited  by  the  many  processes  involved  in  such 
control  has  resulted  in  a  number  of  proposals  for  a 
general  software  architecture  intended  to  provide  an 
efficient  yet  flexible  framework  for  the  organization 
and  interaction  of  relevant  software  components. 

The  Rational  Behavior  Model  (RBM)  has  been 
developed  with  these  requirements  in  mind  and 
consists  of  three  levels,  called  the  Strategic,  the 
Tactical,  and  the  Execution  levels,  respectively. 

Each  level  reflects  computations  supporting  the 
solution  to  the  global  control  problem  based  on 
different  abstraction  mechanisms.  The  unique  con¬ 
tribution  of  the  RBM  architecture  is  the  idea  of 
specifying  different  programming  paradigms  to 
realize  each  software  level.  Specifically,  RBM  uses 
rule-based  programming  for  the  Strategic  level, 
thereby  permitting  field  reconfiguration  of  missions 
by  a  mission  specialist  without  reprogramming  at 
lower  levels.  The  Tactical  level  realizes  vehicle 
behaviors  as  the  methods  of  software  objects  pro¬ 
grammed  in  an  object-based  language  such  as  Ada. 
These  behaviors  are  initiated  by  rule  satisfaction  at 
the  Strategic  level,  thereby  rationalizing  their  inter¬ 
action.  The  Execution  level  is  programmed  in  any 
imperative  language  capable  of  supporting  efficient 
execution  of  real-time  control  of  the  underlying 
vehicle  hardware. 

Supervised  autonomous  control,  shared  control,  and 
teleoperation  for  space  servicing 

94N 11577 

BACKES,  PAUL  G.  In  NASA.  Johnson  Space  Cen¬ 
ter,  Sixth  Annual  Workshop  on  Space  Operations 
Applications  and  Research  (SOAR  1992),  Volume  2 
p  720-731  (SEE  N94-11527  01-99)  Sponsored  by 
NASA,  Washington  Avail:  CASI  HC  A03/MF  A03 
A  local-remote  telerobot  system  for  single-  and 
dual-arm  supervised  autonomy,  shared  control,  and 
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teleoperation  has  been  demonstrated.  The  system  is 
composed  of  two  distinct  parts:  the  local  site,  where 
the  operator  resides;  and  the  remote  site,  where  the 
robots  reside.  The  system  could  be  further  separated 
into  dual  local  sites  communicating  with  a  common 
remote  site.  This  is  valuable  for  potential  space 
missions  where  a  space  based  robotic  system  may 
be  controlled  either  by  a  space  based  operator  or  by 
a  ground  based  operator.  Also,  multiple  modes  of 
control  integrated  into  a  common  system  are  valu¬ 
able  for  satisfying  different  servicing  scenarios.  The 
remote  site  single  arm  control  system  is  described, 
and  its  parameterization  for  different  supervised 
autonomous  control,  shared  control,  and  tele-opera¬ 
tion  tasks  are  given.  Experimental  results  are  also 
given  for  selected  tasks.  The  tasks  include  compli¬ 
ant  grasping,  orbital  replacement  unit  changeout, 
bolt  seating  and  turning,  electronics  card  removal 
and  insertion,  and  door  opening. 

The  development  of  an  interactive  synthesis  tool 
for  intelligent  controllers  of  modular,  reconfigurable 
robots 

94N 10898 

AMBROSE,  CATHERINE  GLAUBER  Avail: 

Univ.  Microfilms  Order  No.  DA9309115 
Intelligent  controller  design  for  modular,  reconfigur¬ 
able  robot  architectures  was  investigated.  In  robotic 
systems  available  today,  the  controller  is  the  main 
limiting  factor  for  enhanced  performance.  Numerous 
researchers  are  studying  new  controller  architectures 
to  allow  for  the  implementation  of  advanced  soft¬ 
ware,  but  few  have  targeted  their  research  at  recon¬ 
figurable  designs.  Focus  was  on  the  development  of 
a  modular  controller  architecture  that  can  be  recon¬ 
figured  to  meet  the  changing  needs  of  new  manipu¬ 
lators  and  new  tasks.  The  dissertation  work  was 
divided  into  two  parallel  lines  of  research.  First,  a 
road  map  assessing  the  required  technology  for  an 
intelligent  controller  was  created.  A  complete  model 
of  robotic  manipulators  was  developed  which  in¬ 
cluded  the  arm  dynamics  and  the  dynamics  of  the 
joint  actuator  systems.  This  model  was  used  to 
determine  the  desired  update  rate  for  command 
signals  from  the  robot  controller.  Second,  a  synthe¬ 
sis  package  was  developed  on  a  Silicon  Graphics 
workstation  to  simulate  controller  architectures.  This 
package  allows  the  user  to  assemble  computer  com¬ 
ponents  and  select  controller  algorithms,  and  then 
evaluates  the  effectiveness  of  the  design.  Bottle¬ 
necks  are  easily  identified  and  improved  architec¬ 
tures  can  be  rapidly  created.  This  synthesis  package 
was  used  to  underscore  the  issues  discussed  in  the 
road  map. 

Telerobot  control  system 

94N 10670 

BACKES,  PAUL  G.;  TSO,  KAM  S.  (Jet  Propulsion 
Lab.,  California  Inst,  of  Tech.,  Pasadena.);  (Jet 
Propulsion  Lab.,  California  Inst,  of  Tech.,  Pasade¬ 
na.)  Filed  9  May  1991  Supersedes  N91-32509  (29  - 
24,  p  4028)  Avail:  US  Patent  and  Trademark  Of¬ 
fice 


This  invention  relates  to  an  operator  interface  for 
controlling  a  telerobot  to  perform  tasks  in  a  poorly 
modeled  environment  and/or  within  unplanned  sce¬ 
narios.  The  telerobot  control  system  includes  a 
remote  robot  manipulator  linked  to  an  operator 
interface.  The  operator  interface  includes  a  setup 
terminal,  simulation  terminal,  and  execution  termi¬ 
nal  for  the  control  of  the  graphics  simulator  and 
local  robot  actuator  as  well  as  the  remote  robot 
actuator.  These  terminals  may  be  combined  in  a 
single  terminal.  Complex  tasks  are  developed  from 
sequential  combinations  of  parameterized  task  prim¬ 
itives  and  recorded  teleoperations,  and  are  tested  by 
execution  on  a  graphics  simulator  and/or  local  robot 
actuator,  together  with  adjustable  time  delays.  The 
novel  features  of  this  invention  include  the  shared 
and  supervisory  control  of  the  remote  robot  manipu¬ 
lator  via  operator  interface  by  pretested  complex 
tasks  sequences  based  on  sequences  of 
parameterized  task  primitives  combined  with  further 
teleoperation  and  run-time  binding  of  parameters 
based  on  task  context. 

Telerobotic  control  of  a  mobile  coordinated  robotic 

server 

94N 10345 

LEE,  GORDON  Avail:  CASI  HC  A04/MF  A01 
The  annual  report  on  telerobotic  control  of  a  mobile 
coordinated  robotic  server  is  presented.  The  goal  of 
this  effort  is  to  develop  advanced  control  methods 
for  flexible  space  manipulator  systems.  As  such,  an 
adaptive  fuzzy  logic  controller  was  developed  in 
which  model  structure  as  well  as  parameter  con¬ 
straints  are  not  required  for  compensation.  The  work 
builds  upon  previous  work  on  fuzzy  logic  control¬ 
lers.  Fuzzy  logic  controllers  have  been  growing  in 
importance  in  the  field  of  automatic  feedback  con¬ 
trol.  Hardware  controllers  using  fuzzy  logic  have 
become  available  as  an  alternative  to  the  traditional 
PID  controllers.  Software  has  also  been  introduced 
to  aid  in  the  development  of  fuzzy  logic  rule-bases. 
The  advantages  of  using  fuzzy  logic  controllers 
include  the  ability  to  merge  the  experience  and 
intuition  of  expert  operators  into  the  rule-base  and 
that  a  model  of  the  system  is  not  required  to  con¬ 
struct  the  controller.  A  drawback  of  the  classical 
fuzzy  logic  controller,  however,  is  the  many  param¬ 
eters  needed  to  be  turned  off-line  prior  to  applica¬ 
tion  in  the  closed-loop.  In  this  report,  an  adaptive 
fuzzy  logic  controller  is  developed  requiring  no 
system  model  or  model  structure.  The  rule-base  is 
defined  to  approximate  a  state-feedback  controller 
while  a  second  fuzzy  logic  algorithm  varies,  on-line, 
parameters  of  the  defining  controller.  Results  indi¬ 
cate  the  approach  is  viable  for  on-line  adaptive 
control  of  systems  when  the  model  is  too  complex 
or  uncertain  for  application  of  other  more  classical 
control  techniques. 

Position  and  force  control  of  a  vehicle  with  two  or 
more  steerable  drive  wheels 

94N 10269 

REISTER,  D.  B.;  UNSEREN,  M.  A.  Avail:  CASI 
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When  a  vehicle  with  two  or  more  steerable  drive 
wheels  is  traveling  in  a  circle,  the  motion  of  the 
wheels  is  constrained.  The  wheel  translational  ve¬ 
locity  divided  by  the  radius  to  the  center  of  rotation 
must  be  the  same  for  all  wheels.  When  the  drive 
wheels  are  controlled  independently  using  position 
control,  the  motion  of  the  wheels  may  violate  the 
constraints  and  the  wheels  may  slip.  Consequently, 
substantial  errors  can  occur  in  the  orientation  of  the 
vehicle.  A  vehicle  with  N  drive  wheels  has  (N  -  1) 
constraints  and  one  degree  of  freedom.  We  have 
developed  a  new  approach  to  the  control  of  a  vehi¬ 
cle  with  N  steerable  drive  wheels.  The  novel  aspect 
of  our  approach  is  the  use  of  force  control.  To  con¬ 
trol  the  vehicle,  we  have  one  degree  of  freedom  for 
the  position  on  the  circle  and  (N  -  1)  forces  that  can 
be  used  to  reduce  errors.  Recently,  Kankaanranta 
and  Koivo  developed  a  control  architecture  that 
allows  the  force  and  position  degrees  of  freedom  to 
be  decoupled.  In  the  work  of  Kankaanranta  and 
Koivo,  the  force  is  an  exogenous  input.  We  have 
made  the  force  endogenous  by  defining  the  force  in 
terms  of  the  errors  in  satisfying  the  rigid  body  kine¬ 
matic  constraints.  We  have  applied  the  control  ar¬ 
chitecture  to  the  HERMIES-3  robot  and  have  mea¬ 
sured  a  dramatic  reduction  in  error  (more  than  a 
factor  of  20)  compared  to  motions  without  force 
control. 

Real-time  qualitative  reasoning  for  telerobotic 

systems 

93N321 16 

PIN,  EANCOIS  G.  In  NASA.  Lyndon  B.  Johnson 
Space  Center,  The  Sixth  Annual  Workshop  on 
Space  Operations  Applications  and  Research 
(SOAR  1992)  p  173  (SEE  N93-32097  12-99) 

Avail:  CASI  HC  A01/MF  A04 
This  paper  discusses  the  sensor-based  telerobotic 
driving  of  a  car  in  a-priori  unknown  environments 
using  'human-like'  reasoning  schemes  implemented 
on  custom-designed  VLSI  fuzzy  inferencing  boards. 
These  boards  use  the  Fuzzy  Set  theoretic  framework 
to  allow  very  vast  (30  kHz)  processing  of  full  sets 
of  information  that  are  expressed  in  qualitative  form 
using  membership  functions.  The  sensor-based  and 
fuzzy  inferencing  system  was  incorporated  on  an 
outdoor  test-bed  platform  to  investigate  two  control 
modes  for  driving  a  car  on  the  basis  of  very  sparse 
and  imprecise  range  data.  In  the  first  mode,  the  car 
navigates  fully  autonomously  to  a  goal  specified  by 
the  operator,  while  in  the  second  mode,  the  system 
acts  as  a  telerobotic  driver's  aid  providing  the  driver 
with  linguistic  (fuzzy)  commands  to  turn  left  or 
right,  speed  up,  slow  down,  stop,  or  back  up  de¬ 
pending  on  the  obstacles  perceived  by  the  sensors. 
Indoor  and  outdoor  experiments  with  both  modes  of 
control  are  described  in  which  the  system  uses  only 
three  acoustic  range  (sonar)  sensor  channels  to  per¬ 
ceive  the  environment.  Sample  results  are  presented 
that  illustrate  the  feasibility  of  developing  autono¬ 
mous  navigation  modules  and  robust, 
safety-enhancing  driver's  aids  for  telerobotic  sys¬ 


tems  using  the  new  fuzzy  inferencing  VLSI  hard¬ 
ware  and  'human-like'  reasoning  schemes. 

TeleOperator/TelePresence  System  (TOPS)  Concept 
Verification  Model  (CVM)  development 

93N321 12 

SHIMAMOTO,  MIKE  S.  In  NASA.  Lyndon  B. 
Johnson  Space  Center,  The  Sixth  Annual  Workshop 
on  Space  Operations  Applications  and  Research 
(SOAR  1992)  p  149-155  (SEE  N93-32097  12-99) 
Avail:  CASI  HC  A02/MF  A04 
The  development  of  an  anthropomorphic,  undersea 
manipulator  system,  the  TeleOperator/telePresence 
System  (TOPS)  Concept  Verification  Model  (CVM) 
is  described.  The  TOPS  system's  design  philosophy, 
which  results  from  NRaD's  experience  in  undersea 
vehicles  and  manipulator  systems  development  and 
operations,  is  presented.  The  TOPS  design  ap¬ 
proach,  task  teams,  manipulator,  and  vision  system 
development  and  results,  conclusions,  and  recom¬ 
mendations  are  presented. 

Air  Force  construction  automation/robotics 

93N321 10 

NEASE,  A.  D.;  ALEXANDER,  E.  F.  In  NASA. 
Lyndon  B.  Johnson  Space  Center,  The  Sixth  Annual 
Workshop  on  Space  Operations  Applications  and 
Research  (SOAR  1992)  p  121-130  (SEE  N93-32097 
12-99)  Avail:  CASI  HC  A02/MF  A04 
The  Air  Force  has  several  missions  which  generate 
unique  requirements  that  are  being  met  through  the 
development  of  construction  robotic  technology. 

One  especially  important  mission  will  be  the  con¬ 
duct  of  Department  of  Defense  (DOD)  space  activ¬ 
ities.  Space  operations  and  other  missions  place 
construction/repair  equipment  operators  in  danger¬ 
ous  environments  and  potentially  harmful  situations. 
Additionally,  force  reductions  require  that  human 
resources  be  leveraged  to  the  maximum  extent  pos¬ 
sible,  and  more  stringent  construction  repair  require¬ 
ments  push  for  increased  automation.  To  solve  these 
problems,  the  U.S.  Air  Force  is  undertaking  a  re¬ 
search  and  development  effort  at  Tyndall  AFB,  FL, 
to  develop  robotic  construction/repair  equipment. 
This  development  effort  involves  the  following 
technologies:  teleoperation,  telerobotics,  construc¬ 
tion  operations  (excavation,  grading,  leveling,  tool 
change),  robotic  vehicle  communications,  vehicle 
navigation,  mission/vehicle  task  control  architecture, 
and  associated  computing  environment.  The  ulti¬ 
mate  goal  is  the  fielding  of  a  robotic  repair  capabili¬ 
ty  operating  at  the  level  of  supervised  autonomy. 
This  paper  will  discuss  current  and  planned  efforts 
in  space  construction/repair,  explosive  ordnance 
disposal,  hazardous  waste  cleanup,  and  fire  fighting. 

Interactive  and  cooperative  sensing  and  control  for 
advanced  teleopcration 

93N32108 

LEE,  SUKHAN  In  NASA.  Lyndon  B.  Johnson 
Space  Center,  The  Sixth  Annual  Workshop  on 
Space  Operations  Applications  and  Research 
(SOAR  1992)  p  104-115  (SEE  N93-32097  12-99) 
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Avail:  CASI  HC  A03/MF  A04 
This  paper  presents  the  paradigm  of  interactive  and 
cooperative  sensing  and  control  as  a  fundamental 
mechanism  of  integrating  and  fusing  the  strengths 
of  man  and  machine  for  advanced  teleoperation. 

The  interactive  and  cooperative  sensing  and  control 
is  considered  as  an  extended  and  generalized  form 
of  traded  and  shared  control.  The  emphasis  of  inter¬ 
active  and  cooperative  sensing  and  control  is  given 
to  the  distribution  of  mutually  nonexclusive 
subtasks  to  man  and  machine,  the  interactive  invo¬ 
cation  of  subtasks  under  the  man/machine  symbiotic 
relationship,  and  the  fusion  of  information  and 
decisionmaking  between  man  and  machine  accord¬ 
ing  to  their  confidence  measures.  The  proposed 
interactive  and  cooperative  sensing  and  control 
system  is  composed  of  such  major  functional  blocks 
as  the  logical  sensor  system,  the  sensor-based  local 
autonomy,  the  virtual  environment  formation,  and 
the  cooperative  decision-making  between  man  and 
machine.  The  Sensing-Knowledge-Command  (SKC) 
fusion  network  is  proposed  as  a  fundamental  archi¬ 
tecture  for  implementing  cooperative  and  interactive 
sensing  and  control.  Simulation  results  are  shown. 

Integration  of  advanced  teleoperation  technologies 
for  control  of  space  robots 

93N32107 

STAGNARO,  MICHAEL  J.  In  its  The  Sixth  Annual 
Workshop  on  Space  Operations  Applications  and 
Research  (SOAR  1992)  p  94-103  (SEE  N93-32097 
12-99)  Avail:  CASI  HC  A02/MF  A04 
Teleoperated  robots  require  one  or  more  humans  to 
control  actuators,  mechanisms,  and  other  robot 
equipment  given  feedback  from  onboard  sensors.  To 
accomplish  this  task,  the  human  or  humans  require 
some  form  of  control  station.  Desirable  features  of 
such  a  control  station  include  operation  by  a  single 
human,  comfort,  and  natural  human  interfaces  (visu¬ 
al,  audio,  motion,  tactile,  etc.).  These  interfaces 
should  work  to  maximize  performance  of  the  hu¬ 
man/robot  system  by  streamlining  the  link  between 
human  brain  and  robot  equipment.  This  paper  de¬ 
scribes  development  of  a  control  station  testbed 
with  the  characteristics  described  above.  Initially, 
this  testbed  will  be  used  to  control  two  teleoperated 
robots.  Features  of  the  robots  include  anthropomor¬ 
phic  mechanisms,  slaving  to  the  testbed,  and  deliv¬ 
ery  of  sensory  feedback  to  the  testbed.  The  testbed 
will  make  use  of  technologies  such  as  helmet 
mounted  displays,  voice  recognition,  and  exoskele¬ 
ton  masters.  It  will  allow  tor  integration  and  testing 
of  emerging  telepresence  technologies  along  with 
techniques  for  coping  with  control  link  time  delays. 
Systems  developed  from  this  testbed  could  be  ap¬ 
plied  to  ground  control  of  space  based  robots.  Dur¬ 
ing  man-tended  operations,  the  Space  Station  Free¬ 
dom  may  benefit  from  ground  control  of  IVA  or 
EVA  robots  with  science  or  maintenance  tasks. 
Planetary  exploration  may  also  find  advanced 
teleoperation  systems  to  be  very  useful. 


Man-machine  cooperation  in  advanced  teleoperation 

93N32106 

FIORINI,  PAOLO;  DAS,  HARI;  LEE,  SUKHAN  In 
NASA.  Lyndon  B.  Johnson  Space  Center,  The  Sixth 
Annual  Workshop  on  Space  Operations  Applica¬ 
tions  and  Research  (SOAR  1992)  p  87-93  (SEE 
N93-32097  12-99)  Avail:  CASI  HC  A02/MF  A04 
Teleoperation  experiments  at  JPL  have  shown  that 
advanced  features  in  a  telerobotic  system  are  a 
necessary  condition  for  good  results,  but  that  they 
are  not  sufficient  to  assure  consistently  good  perfor¬ 
mance  by  the  operators.  Two  or  three  operators  are 
normally  used  during  training  and  experiments  to 
maintain  the  desired  performance.  An  alternative  to 
this  multi-operator  control  station  is  a  man-machine 
interface  embedding  computer  programs  that  can 
perform  some  of  the  operator's  functions.  In  this 
paper  we  present  our  first  experiments  with  these 
concepts,  in  which  we  focused  on  the  areas  of 
real-time  task  monitoring  and  interactive  path  plan¬ 
ning.  In  the  first  case,  when  performing  a  known 
task,  the  operator  has  an  automatic  aid  for  setting 
control  parameters  and  camera  views.  In  the  second 
case,  an  interactive  path  planner  will  rank  different 
path  alternatives  so  that  the  operator  will  make  the 
correct  control  decision.  The  monitoring  function 
has  been  implemented  with  a  neural  network  doing 
the  real-time  task  segmentation.  The  interactive  path 
planner  was  implemented  for  redundant  manipula¬ 
tors  to  specify  arm  configurations  across  the  desired 
path  and  satisfy  geometric,  task,  and  performance 
constraints. 

Supervisory  autonomous  local-remote  control 
system  design:  Near-term  and  far-term  applications 

93N32100 

ZIMMERMAN,  WAYNE;  BACKES,  PAUL  In 
NASA.  Lyndon  B.  Johnson  Space  Center,  The  Sixth 
Annual  Workshop  on  Space  Operations  Applica¬ 
tions  and  Research  (SOAR  1992)  p  28-40  (SEE 
N93-32097  12-99)  Avail:  CASI  HC  A03/MF  A04 
The  JPL  Supervisory  Telerobotics  Laboratory 
(STELER)  has  developed  a  unique  local-remote 
robot  control  architecture  which  enables  manage¬ 
ment  of  intermittent  bus  latencies  and  communica¬ 
tion  delays  such  as  those  expected  for 
ground-remote  operation  of  Space  Station  robotic 
systems  via  the  TDRSS  communication  platform. 

At  the  local  site,  the  operator  updates  the  work  site 
world  model  using  stereo  video  feedback  and  a 
model  overlay/fitting  algorithm  which  outputs  the 
location  and  orientation  of  the  object  in  free  space. 
That  information  is  relayed  to  the  robot  User  Macro 
Interface  (UMI)  to  enable  programming  of  the  robot 
control  macros.  The  operator  can  then  employ  either 
manual  teleoperation,  shared  control,  or  supervised 
autonomous  control  to  manipulate  the  object  under 
any  degree  of  time-delay.  The  remote  site  performs 
the  closed  loop  force/torque  control,  task  monitor¬ 
ing,  and  reflex  action.  This  paper  describes  the 
STELER  local-remote  robot  control  system,  and 
further  describes  the  near-term  planned  Space  Sta¬ 
tion  applications,  along  with  potential  far-term  ap- 
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plications  such  as  telescience,  autonomous  docking, 
and  Lunar/Mars  rovers. 

A  vision  system  planner  for  increasing  the  autono¬ 
my  of  the  Extravehicular  Activity  Helper/Retriever 
93N31844 

MAGEE,  MICHAEL  Avail:  CASI  HC  A04/ 

MF  A01 

The  Extravehicular  Activity  Retriever  (EVAR)  is  a 
robotic  device  currently  being  developed  by  the 
Automation  and  Robotics  Division  at  the  NASA 
Johnson  Space  Center  to  support  activities  in  the 
neighborhood  of  the  Space  Shuttle  or  Space  Station 
Freedom.  As  the  name  implies,  the  Retriever's  pri¬ 
mary  function  will  be  to  provide  the  capability  to 
retrieve  tools  and  equipment  or  other  objects  which 
have  become  detached  from  the  spacecraft,  but  it 
will  also  be  able  to  rescue  a  crew  member  who  may 
have  become  inadvertently  de-tethered.  Later  goals 
will  include  cooperative  operations  between  a  crew 
member  and  the  Retriever  such  as  fetching  a  tool 
that  is  required  for  servicing  or  maintenance  opera¬ 
tions.  This  paper  documents  a  preliminary  design 
for  a  Vision  System  Planner  (VSP)  for  the  EVAR 
that  is  capable  of  achieving  visual  objectives  pro¬ 
vided  to  it  by  a  high  level  task  planner.  Typical 
commands  which  the  task  planner  might  issue  to  the 
VSP  relate  to  object  recognition,  object  location 
determination,  and  obstacle  detection.  Upon  receiv¬ 
ing  a  command  from  the  task  planner,  the  VSP  then 
plans  a  sequence  of  actions  to  achieve  the  specified 
objective  using  a  model-based  reasoning  approach. 
This  sequence  may  involve  choosing  an  appropriate 
sensor,  selecting  an  algorithm  to  process  the  data, 
reorienting  the  sensor,  adjusting  the  effective  resolu¬ 
tion  of  the  image  using  lens  zooming  capability, 
and/or  requesting  the  task  planner  to  reposition  the 
EVAR  to  obtain  a  different  view  of  the  object.  An 
initial  version  of  the  Vision  System  Planner  which 
realizes  the  above  capabilities  using  simulated  imag¬ 
es  has  been  implemented  and  tested.  The  remaining 
sections  describe  the  architecture  and  capabilities  of 
the  VSP  and  its  relationship  to  the  high  level  task 
planner.  In  addition,  typical  plans  that  are  generated 
to  achieve  visual  goals  for  various  scenarios  are 
discussed.  Specific  topics  to  be  addressed  will  in¬ 
clude  object  search  strategies,  repositioning  of  the 
EVAR  to  improve  the  quality  of  information  ob¬ 
tained  from  the  sensors,  and  complementary  usage 
of  the  sensors  and  redundant  capabilities. 

Supervised  autonomy  in  tclcnobotics 

93N30326 

BURTNYK,  N.;  BASRAN,  J.  (Ottawa  Univ.,  Ontar¬ 
io.)  In  Engineering  Inst,  of  Canada,  Canadian  Con¬ 
ference  on  Electrical  and  Computer  Engineering, 
Volumes  1  and  2  7p  (SEE  N93-30215  11-31) 

Avail:  Engineering  Inst,  of  Canada,  2050  rue 
Mansfield,  Suite  700,  Montreal,  Quebec  H3A  1Z2 
Canada 

Advanced  autonomous  robots  are  envisaged  for 
applications  in  harsh,  demanding  or  dangerous  envi¬ 
ronments  such  as  nuclear,  underwater  operations, 


space,  mining  and  tunneling,  civil  engineering  and 
construction,  etc.  A  major  challenge  in  telerobotics 
development  is  to  incorporate  the  robot's  capability 
for  autonomous  operation  wherever  possible.  A 
supervisory  control  concept  for  exploiting  autono¬ 
mous  functionality  in  an  unstructured  environment 
is  presented.  Through  an  effective  man-machine 
dialogue,  the  operator  uses  human  perceptual  and 
planning  abilities  to  assist  the  robot  in  acquiring  a 
model  of  its  environment.  Once  this  structure  is 
acquired,  the  operator  instructs  the  robot  on  how  to 
perform  a  task  as  a  sequence  of  actions  based  on 
generic  skills.  A  discussion  of  the  technical  solu¬ 
tions  that  are  essential  to  implement  this  concept 
and  the  results  of  a  study  to  develop  a  robot  operat¬ 
ing  under  supervised  autonomy  are  presented. 

A  functional  system  architecture  for  fully 
autonomous  robot 

93N3031 1 

KALAYCIOGLU,  S.  In  Engineering  Inst,  of  Can¬ 
ada,  Canadian  Conference  on  Electrical  and  Com¬ 
puter  Engineering,  Volumes  1  and  2  4  p  (SEE 
N93-30215  11-31)  Avail:  Engineering  Inst,  of  Can¬ 
ada,  2050  rue  Mansfield,  Suite  700,  Montreal,  Que¬ 
bec  H3A  1Z2  Canada 

The  Mobile  Servicing  System  (MSS)  Autonomous 
Robotics  Program  intends  to  define  and  plan  the 
development  of  technologies  required  to  provide  a 
supervised  autonomous  operation  capability  for  the 
Special  Purpose  Dexterous  Manipulator  (SPDM)  on 
the  MSS.  The  operational  functions  for  the  SPDM 
to  perform  the  required  tasks,  both  in  fully  autono¬ 
mous  or  supervised  modes,  are  identified.  Function¬ 
al  decomposition  is  performed  using  a  graphics 
oriented  methodology  called  Structural  Analysis 
Design  Technique.  This  process  defines  the  func¬ 
tional  architecture  of  the  system,  the  types  of  data 
required  to  support  its  functionality,  and  the  control 
processes  that  need  to  be  emplaced.  On  the  basis  of 
the  functional  decomposition,  a  technology  break¬ 
down  structure  is  also  developed.  A  preliminary 
estimate  of  the  status  and  maturity  of  each  relevant 
technology  is  made,  based  on  this  technology  break¬ 
down.  The  developed  functional  hierarchy  is  found 
to  be  very  effective  for  a  robotic  system  with  any 
level  of  autonomy.  Moreover,  this  hierarchy  can 
easily  be  applied  to  an  existing  very  low  level  au¬ 
tonomous  system  and  can  provide  a  smooth  transi¬ 
tion  towards  a  higher  degree  of  autonomy.  The 
effectiveness  of  the  developed  functional  hierarchy 
will  also  play  a  very  significant  role  both  in  the 
system  design  as  well  as  in  the  development  of  the 
control  hierarchy. 

Input  relegation  control  for  gross  motion  of  a 
kinematically  redundant  manipulator 

93N30210 

UNSEREN,  M.  A.  Presented  at  the  1993  Institute 
of  Electrical  and  Electronics  Engineers/Robotics 
Society  of  Japan  (IEERSJ)  International  Conference 
on  Intelligent  Robots  and  Systems  (IROS),  Yoko¬ 
hama,  Japan,  26-30  Jul.  1993  Avail:  CASI  HC 
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A03/MF  A01 

This  paper  proposes  a  method  for  resolving  the 
kinematic  redundancy  of  a  serial  link  manipulator 
moving  in  a  three-dimensional  workspace.  The 
underspecified  problem  of  solving  for  the  joint 
velocities  based  on  the  classical  kinematic  velocity 
model  is  transformed  into  a  well-specified  problem. 
This  is  accomplished  by  augmenting  the  original 
model  with  additional  equations  which  relate  a  new 
vector  variable  quantifying  the  redundant  degrees  of 
freedom  (DOF)  to  the  joint  velocities.  The  resulting 
augmented  system  yields  a  well  specified  solution 
for  the  joint  velocities.  Methods  for  selecting  the 
redundant  DOF  quantifying  variable  and  the  trans¬ 
formation  matrix  relating  it  to  the  joint  velocities 
are  presented  so  as  to  obtain  a  minimum  Euclidean 
norm  solution  for  the  joint  velocities.  The  results 
obtained  from  experimentally  implementing  the 
proposed  scheme  on  the  CESARM  research  manip¬ 
ulator  are  presented. 

The  planning  coordinator;  A  design  architecture  for 
autonomous  error  recovery  and  on-line  planning  of 
intelligent  tasks 

93N30039 

FARAH,  JEFFREY  J.  Avail:  CASI  HC  A07/ 

MF  A  02 

Developing  a  robust,  task  level,  error  recovery  and 
on-line  planning  architecture  is  an  open  research 
area.  There  is  previously  published  work  on  both 
error  recovery  and  on-line  planning;  however,  none 
incorporates  error  recovery  and  on-line  planning 
into  one  integrated  platform.  The  integration  of 
these  two  functionalities  requires  an  architecture 
that  possesses  the  following  characteristics.  The 
architecture  must  provide  for  the  inclusion  of  new 
information  without  the  destruction  of  existing  in¬ 
formation.  The  architecture  must  provide  for  the 
relating  of  pieces  of  information,  old  and  new,  to 
one  another  in  a  non-trivial  rather  than  trivial  man¬ 
ner  (e.g.,  object  one  is  related  to  object  two  under 
the  following  constraints,  versus,  yes,  they  are  relat¬ 
ed;  no,  they  are  not  related).  Finally,  the  architec¬ 
ture  must  be  not  only  a  stand  alone  architecture,  but 
also  one  that  can  be  easily  integrated  as  a  supple¬ 
ment  to  some  existing  architecture.  This  thesis  pro¬ 
posal  addresses  architectural  development.  Its  intent 
is  to  integrate  error  recovery  and  on-line  planning 
onto  a  single,  integrated,  multi-processor  platform. 
This  intelligent  x-autonomous  platform,  called  the 
Planning  Coordinator,  will  be  used  initially  to  sup¬ 
plement  existing  x-autonomous  systems  and  eventu¬ 
ally  replace  them. 

Application  of  sensors  to  the  control  of  robotic 

systems 

93N30012 

HARRIGAN,  R.  W.  Presented  at  the  50th  Anniver¬ 
sary  Spring  Conference  on  Experimental  Mechanics, 
Dearborn,  MI,  9  Jun.  1993  Avail:  CASI  HC 
A02/MF  A01 

Hazardous  operations  which  in  the  past  have  been 
completed  by  technicians  are  under  increased  scru¬ 


tiny  due  to  high  costs  and  low  productivity  associ¬ 
ated  with  providing  protective  clothing  and  environ¬ 
ments.  As  a  result,  remote  systems  are  needed  to 
accomplish  many  hazardous  materials  handling 
tasks  such  as  the  clean  up  of  waste  sites  in  which 
the  exposure  of  personnel  to  radiation,  chemical, 
explosive,  and  other  hazardous  constituents  is  unac¬ 
ceptable.  Traditional  remote  manual  operations  have 
proven  to  have  very  low  productivity  when  com¬ 
pared  with  unencumbered  humans.  Computer  mod¬ 
els  augmented  by  sensing  and  structured,  modular 
computing  environments  are  proving  to  be  effective 
in  automating  many  unstructured  hazardous  tasks. 

Planning  collision  free  paths  for  two  cooperating 
robots  using  a  divide-and-conquer  C-space  traversal 
heuristic 

93N29771 

WEAVER,  JOHNATHAN  M.  Avail:  CASI  HC 
A08/MF  A02 

A  method  was  developed  to  plan  feasible  and 
obstacle-avoiding  paths  for  two  spatial  robots  work¬ 
ing  cooperatively  in  a  known  static  environment. 
Cooperating  spatial  robots  as  referred  to  herein  are 
robots  which  work  in  6D  task  space  while  simulta¬ 
neously  grasping  and  manipulating  a  common,  rigid 
payload.  The  approach  is  configuration  space 
(c-space)  based  and  performs  selective  rather  than 
exhaustive  c-space  mapping.  No  expensive 
precomputations  are  required.  A  novel,  divide-and- 
conquer  type  of  heuristic  is  used  to  guide  the  selec¬ 
tive  mapping  process.  The  heuristic  does  not  in¬ 
volve  any  robot,  environment,  or  task  specific  as¬ 
sumptions.  A  technique  was  also  developed  which 
enables  solution  of  the  cooperating  redundant  robot 
path  planning  problem  without  requiring  the  use  of 
inverse  kinematics  for  a  redundant  robot.  The  path 
planning  strategy  involves  first  attempting  to  tra¬ 
verse  along  the  configuration  space  vector  from  the 
start  point  towards  the  goal  point.  If  an  unsafe  re¬ 
gion  is  encountered,  an  intermediate  via  point  is 
identified  by  conducting  a  systematic  search  in  the 
hyperplane  orthogonal  to  and  bisecting  the  unsafe 
region  of  the  vector.  This  process  is  repeatedly 
applied  until  a  solution  to  the  global  path  planning 
problem  is  obtained.  The  basic  concept  behind  this 
strategy  is  that  better  local  decisions  at  the  begin¬ 
ning  of  the  trouble  region  may  be  made  if  a  possi¬ 
ble  way  around  the  'center'  of  the  trouble  region  is 
known.  Thus,  rather  than  attempting  paths  which 
look  promising  locally  (at  the  beginning  of  a  trouble 
region)  but  which  may  not  yield  overall  results,  the 
heuristic  attempts  local  strategies  that  appear  prom¬ 
ising  for  circumventing  the  unsafe  region. 

An  autonomous  vision-based  mobile  robot 

93N29082 

BAUMGARTNER,  ERIC  THOMAS  Avail:  Univ. 
Microfilms  Order  No.  DA9308220 
This  dissertation  describes  estimation  and  control 
methods  for  use  in  the  development  of  an  autono¬ 
mous  mobile  robot  for  structured  environments.  The 
navigation  of  the  mobile  robot  is  based  on  precise 
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estimates  of  the  position  and  orientation  of  the  robot 
within  its  environment.  The  extended  Kalman  filter 
algorithm  is  used  to  combine  information  from  the 
robot's  drive  wheels  with  periodic  observations  of 
small,  wall-mounted,  visual  cues  to  produce  the 
precise  position  and  orientation  estimates.  The  visu¬ 
al  cues  are  reliably  detected  by  at  least  one  video 
camera  mounted  on  the  mobile  robot.  Typical  posi¬ 
tion  estimates  are  accurate  to  within  one  inch.  A 
path  tracking  algorithm  is  also  developed  to  follow 
desired  reference  paths  which  are  taught  by  a  hu¬ 
man  operator.  Because  of  the  time-independence  of 
the  tracking  algorithm,  the  speed  that  the  vehicle 
travels  along  the  reference  path  is  specified  inde¬ 
pendent  from  the  tracking  algorithm.  The  estimation 
and  control  methods  have  been  applied  successfully 
to  two  experimental  vehicle  systems.  Finally,  an 
analysis  of  the  linearized  closed-loop  control  system 
is  performed  to  study  the  behavior  and  the  stability 
of  the  system  as  a  function  of  various  control 
parameters. 

Robotics  and  artificial  intelligence  for  hazardous 
environments 

93N29047 

SPELT,  P.  F.  Presented  at  the  17th  Congress  of 
Investigation  Cientifica  Universidad  Interamericana 
De  Puerto  Rico,  San  Juan,  Puerto  Rico,  11-12  Feb. 
1993  Avail:  CASI  HC  A03/MF  A01 
In  our  technological  society,  hazardous  materials 
including  toxic  chemicals,  flammable,  explosive, 
and  radioactive  substances,  and  biological  agents, 
are  used  and  handled  routinely.  Each  year,  many 
workers  who  handle  these  substances  are  accidently 
contaminated,  in  some  cases  resulting  in  injury, 
death,  or  chronic  disabilities.  If  these  hazardous 
materials  could  be  handled  remotely,  either  with  a 
teleoperated  robot  (operated  by  a  worker  in  a  safe 
location)  or  by  an  autonomous  robot,  then  human 
suffering  and  economic  costs  of  accidental  expo¬ 
sures  could  be  dramatically  reduced.  At  present,  it 
is  still  difficult  for  commercial  robotic  technology 
to  completely  replace  humans  involved  in  perform¬ 
ing  complex  work  tasks  in  hazardous  environments. 
The  robotics  efforts  at  the  Center  for  Engineering 
Systems  Advanced  Research  represent  a  significant 
effort  at  contributing  to  the  advancement  of  robotics 
for  use  in  hazardous  environments.  While  this  effort 
is  very  broad-based,  ranging  from  dextrous  manipu¬ 
lation  to  mobility  and  integrated  sensing,  focus  is  on 
machine  learning  and  the  high-level  decision  mak¬ 
ing  needed  for  autonomous  robotics. 

Intelligent  robots  for  planetary  exploration  and 
construction 

93N27980 

ALBUS,  JAMES  S.  In  Arizona  Univ.,  Proceedings 
of  the  Lunar  Materials  Technology  Symposium  15  p 
(SEE  N93-27956  10-91)  Avail:  CASI  HC  A03/ 

MF  A03 

Robots  capable  of  practical  applications  in  planetary 
exploration  and  construction  will  require  realtime 
sensory-interactive  goal-directed  control  systems.  A 


reference  model  architecture  based  on  the  NIST 
Real-time  Control  System  (RCS)  for  real-time  intel¬ 
ligent  control  systems  is  suggested.  RCS  partitions 
the  control  problem  into  four  basic  elements:  be¬ 
havior  generation  (or  task  decomposition),  world 
modeling,  sensory  processing,  and  value  judgment. 

It  clusters  these  elements  into  computational  nodes 
that  have  responsibility  for  specific  subsystems,  and 
arranges  these  nodes  in  hierarchical  layers  such  that 
each  layer  has  characteristic  functionality  and  tim¬ 
ing.  Planetary  exploration  robots  should  have  mo¬ 
bility  systems  that  can  safely  maneuver  over  rough 
surfaces  at  high  speeds.  Walking  machines  and 
wheeled  vehicles  with  dynamic  suspensions  are 
candidates.  The  technology  of  sensing  and  sensory 
processing  has  progressed  to  the  point  where 
real-time  autonomous  path  planning  and  obstacle 
avoidance  behavior  is  feasible.  Map-based  naviga¬ 
tion  systems  will  support  long-range  mobility  goals 
and  plans.  Planetary  construction  robots  must  have 
high  strength-to-weight  ratios  for  lifting  and  posi¬ 
tioning  tools  and  materials  in  six  degrees-of-free- 
dom  over  large  working  volumes.  A  new  generation 
of  cable-suspended  Stewart  platform  devices  and  in¬ 
flatable  structures  are  suggested  for  lifting  and  posi¬ 
tioning  materials  and  structures,  as  well  as  for  exca¬ 
vation,  grading,  and  manipulating  a  variety  of  tools 
and  construction  machinery. 

Architecture  of  autonomous  systems 

93N26047 

DIKSHIT,  PIYUSH;  GUIMARAES,  KATIA; 
RAMAMURTPIY,  MAYA;  AGRAWALA, 

ASHOK;  LARSEN,  RONALD  L.  Avail:  CASI  HC 
A04/MF  A01 

Automation  of  Space  Station  functions  and  activi¬ 
ties,  particularly  those  involving  robotic  capabilities 
with  interactive  or  supervisory  human  control,  is  a 
complex,  multi-disciplinary  systems  design  problem. 
A  wide  variety  of  applications  using  autonomous 
control  can  be  found  in  the  literature,  but  none  of 
them  seem  to  address  the  problem  in  general.  All  of 
them  are  designed  with  a  specific  application  in 
mind.  In  this  report,  an  abstract  model  is  described 
which  unifies  the  key  concepts  underlying  the  de¬ 
sign  of  automated  systems  such  as  those  studied  by 
the  aerospace  contractors.  The  model  has  been  kept 
as  general  as  possible.  The  attempt  is  to  capture  all 
the  key  components  of  autonomous  systems.  With  a 
little  effort,  it  should  be  possible  to  map  the  func¬ 
tions  of  any  specific  autonomous  system  application 
to  the  model  presented  here. 

Real-time  construction  and  rendering  of 
three-dimensional  occupancy  maps 

93N25516 

JONES,  J.  P.  Presented  at  the  11th  Society  of 
Photo-Optical  Instrumentation  Engineering  Appli¬ 
cations  of  Artificial  Intelligence,  Orlando,  FL,  12-16 
Apr.  1993  Avail:  CASI  HC  A02/MF  A01 
This  paper  describes  a  preliminary  sensory  system 
for  real-time  sensor-based  navigation  in  a  three- 
dimensional,  dynamic  environment.  Data  from  a 
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laser  range  camera  are  processed  on  an  iWarp  paral¬ 
lel  computer  to  create  a  3D  occupancy  map.  This 
map  is  rendered  using  raytracing.  The  construction 
and  rendering  consume  less  than  800  milliseconds. 

A  global  path  planning  approach  for  redundant 
manipulators 

93N24656 

SEEREERAM,  SANJEEV;  WEN,  J.  Avail:  CASI 
HC  A04/MF  A01 

A  new  approach  for  global  path  planning  of  redun¬ 
dant  manipulators  is  proposed.  It  poses  the  path 
planning  problem  as  a  finite  time  nonlinear  control 
problem.  The  solution  is  found  by  a  Newton- 
Raphson  type  algorithm.  This  technique  is  capable 
of  handling  various  goal  task  descriptions  as  well  as 
incorporating  both  joint  and  task  space  constraints. 
The  algorithm  has  shown  promising  preliminary 
results  in  planning  joint  path  sequences  for  3R  and 
4R  planar  robots  to  meet  Cartesian  tip  tracking  and 
goal  endpoint  planning.  It  is  robust  with  respect  to 
local  path  planning  problems  such  as  singularity 
considerations  and  local  minimum  problems.  Repet¬ 
itive  joint  path  solutions  for  cyclic  end-effector 
tasks  are  also  generated.  Eventual  goals  of  this 
work  include  implementation  on  full  spatial  robots, 
as  well  as  provision  of  an  interface  for  supervisory 
input  to  aid  in  path  planning  for  more  complex 
problems. 


Transformational  planning  of  reactive 
behavior 

93N24270 

MCDERMOTT,  DREW  Avail:  CASI  HC  A04/ 

MF  A01 

Reactive  plans  are  plans  that  include  steps  for  sens¬ 
ing  the  world  and  coping  with  the  data  so  obtained. 
The  application  of  AI  planning  techniques  to  plans 
of  this  sort  in  a  simple  simulated  world  is  investi¬ 
gated.  To  achieve  fast  reaction  times,  it  is  assumed 
that  the  agent  starts  with  a  default  reactive  plan, 
while  the  planner  attempts  to  improve  it  by  apply¬ 
ing  plan  transformations,  thus  searching  through  the 
space  of  transformed  plans.  When  the  planner  has 
what  it  believes  to  be  a  better  plan,  it  swaps  the 
new  plan  into  the  agent's  controller.  The  plans  are 
written  in  a  reactive  language  that  allows  for  this 
kind  of  swapping.  The  language  allows  for 
concurrency,  and  hence,  truly  'nonlinear'  plans.  The 
planner  evaluates  plans  by  projecting  them,  that  is, 
generating  scenarios  for  how  execution  might  go. 
The  resulting  projections  give  estimates  of  plan 
values,  but  also  provide  clues  to  how  the  plan  might 
be  improved.  These  clues  are  unearthed  by  critics 
that  go  through  the  scenario  sets,  checking  how  the 
world  state  and  the  agent  state  evolved.  The  critics 
suggest  plan  transformations  with  associated  esti¬ 
mates  of  how  much  they  will  improve  the  plan. 

Plan  transformations  must  be  able  to  edit  code  trees 
in  such  a  way  that  the  changes  are  orthogonal  and 
reversible  whenever  possible.  The  system  was  tested 
by  comparing  the  performance  of  the  agent  with 


and  without  planning.  Preliminary  results  allow  us 
to  conclude  that  the  planner  can  be  fast  and  directed 
enough  to  generate  improved  plans  in  a  timely  fash¬ 
ion,  and  that  the  controller  can  often  cope  with  a 
sudden  shift  of  plan. 

Kinematics  and  dynamics  of  robotic  systems  with 
multiple  closed  loops 

93N24218 

ZHANG,  CHANG-DE  Avail:  Univ.  Microfilms 
Order  No.  DA9238371 

The  kinematics  and  dynamics  of  robotic  systems 
with  multiple  closed  loops,  such  as  Stewart  plat¬ 
forms,  walking  machines,  and  hybrid  manipulators, 
are  studied.  In  the  study  of  kinematics,  focus  is  on 
the  closed-form  solutions  of  the  forward  position 
analysis  of  different  parallel  systems.  A  closed-form 
solution  means  that  the  solution  is  expressed  as  a 
polynomial  in  one  variable.  If  the  order  of  the  poly¬ 
nomial  is  less  than  or  equal  to  four,  the  solution  has 
analytical  closed-form.  First,  the  conditions  of  ob¬ 
taining  analytical  closed-form  solutions  are  studied. 
For  a  Stewart  platform,  the  condition  is  found  to  be 
that  one  rotational  degree  of  freedom  of  the  output 
link  is  decoupled  from  the  other  five.  Based  on  this 
condition,  a  class  of  Stewart  platforms  which  has 
analytical  closed-form  solution  is  formulated.  Con¬ 
ditions  of  analytical  closed-form  solution  for  other 
parallel  systems  are  also  studied.  Closed-form  solu¬ 
tions  of  forward  kinematics  for  walking  machines 
and  multi-fingered  grippers  are  then  studied.  For  a 
parallel  system  with  three  three-degree-of-freedom 
subchains,  there  are  84  possible  ways  to  select  six 
independent  joints  among  nine  joints.  These  84 
ways  can  be  classified  into  three  categories:  Catego¬ 
ry  3:3:0,  Category  3:2:1,  and  Category  2:2:2.  It  is 
shown  that  the  first  category  has  no  solutions;  the 
solutions  of  the  second  category  have  analytical 
closed-form;  and  the  solutions  of  the  last  category 
are  higher  order  polynomials.  The  study  is  then 
extended  to  a  nearly  general  Stewart  platform.  The 
solution  is  a  20th  order  polynomial  and  the  Stewart 
platform  has  a  maximum  of  40  possible  configura¬ 
tions.  Also,  the  study  is  extended  to  a  new  class  of 
hybrid  manipulators  which  consists  of  two  serially 
connected  parallel  mechanisms.  In  the  study  of 
dynamics,  a  computationally  efficient  method  for 
inverse  dynamics  of  manipulators  based  on  the 
virtual  work  principle  is  developed.  Although  this 
method  is  comparable  with  the  recursive 
Newton-Euler  method  for  serial  manipulators,  its 
advantage  is  more  noteworthy  when  applied  to 
parallel  systems.  An  approach  of  inverse  dynamics 
of  a  walking  machine  is  also  developed,  which 
includes  inverse  dynamic  modeling,  foot  force  dis¬ 
tribution,  and  joint  force/torque  allocation.  A  new 
approach  is  then  proposed  to  determine  the  foot 
distribution  of  planar  walking  gaits  based  on  two 
optimum  criteria.  A  computational  scheme  is  then 
developed  to  determine  the  joint  force/torque  alloca¬ 
tion,  on  the  basis  of  the  virtual  work  principle.  As 
an  illustration,  the  inverse  dynamic  modeling  of  a 
quadruped  with  pantograph  legs  is  derived.  The 
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inverse  dynamics  of  this  quadruped  walking  in 
different  wave  gaits  is  then  studied. 

Trajectory  control  of  an  articulated  robot  with  a 
parallel  drive  arm  based  on  splines  under  tension 

93N24184 

YI,  SEUNG-JONG  Avail:  Univ.  Microfilms  Order 
No.  DA9238369 

Today's  industrial  robots  controlled  by  mini/micro 
computers  are  basically  simple  positioning  devices. 
The  positioning  accuracy  depends  on  the  mathe¬ 
matical  description  of  the  robot  configuration  to 
place  the  end-effector  at  the  desired  position  and 
orientation  within  the  workspace  and  on  following 
the  specified  path  which  requires  the  trajectory 
planner.  In  addition,  the  consideration  of  joint  ve¬ 
locity,  acceleration,  and  jerk  trajectories  are  essen¬ 
tial  for  trajectory  planning  of  industrial  robots  to 
obtain  smooth  operation.  The  newly  designed  6 
DOF  articulated  robot  with  a  parallel  drive  arm 
mechanism  which  permits  the  joint  actuators  to  be 
placed  in  the  same  horizontal  line  to  reduce  the  arm 
inertia  and  to  increase  load  capacity  and  stiffness  is 
selected.  First,  the  forward  kinematic  and  inverse 
kinematic  problems  are  examined.  The  forward 
kinematic  equations  are  successfully  derived  based 
on  Denavit-Hartenberg  notation  with  independent 
joint  angle  constraints.  The  inverse  kinematic  prob¬ 
lems  are  solved  using  the  arm-wrist  partitioned 
approach  with  independent  joint  angle  constraints. 
Three  types  of  curve  fitting  methods  used  in  trajec¬ 
tory  planning,  i.e.,  certain  degree  polynomial  func¬ 
tions,  cubic  spline  functions,  and  cubic  spline  func¬ 
tions  under  tension,  are  compared  to  select  the  best 
possible  method  to  satisfy  both  smooth  joint  tra¬ 
jectories  and  positioning  accuracy  for  a  robot  trajec¬ 
tory  planner.  Cubic  spline  functions  under  tension  is 
the  method  selected  for  the  new  trajectory  planner. 
This  method  is  implemented  for  a  6  DOF  articulat¬ 
ed  robot  with  a  parallel  drive  arm  mechanism  to 
improve  the  smoothness  of  the  joint  trajectories  and 
the  positioning  accuracy  of  the  manipulator.  Also, 
this  approach  is  compared  with  existing  trajectory 
planners,  4-3-4  polynomials  and  cubic  spline  func¬ 
tions,  via  circular  arc  motion  simulations.  The  new 
trajectory  planner  using  cubic  spline  functions  under 
tension  is  implemented  into  the  microprocessor 
based  robot  controller  and  motors  to  produce  com¬ 
bined  arc  and  straight-line  motion.  The  simulation 
and  experiment  show  interesting  results  by  demon¬ 
strating  smooth  motion  in  both  acceleration  and  jerk 
and  significant  improvements  of  positioning  accura¬ 
cy  in  trajectory  planning. 

Perception,  scene  reconstruction  and  world 
modeling  for  unmanned  underwater  vehicles 

93N23443 

MOORE,  JOHN,  JR.  Workshop  held  in  Cambridge, 
MA,  26-27  Jan.  1993  Sponsored  by  National  Sea 
Grant  Coll.  Program,  Silver  Spring,  MD  Avail: 
CASI  HC  A03/MF  A01 

The  workshop  addresses  the  sensory  and  perceptual 
issues  associated  with  the  operation  of  intelligent 


systems  in  an  underwater  environment.  Topics 
range  from  low-level  processing,  including  novel 
hardware  approaches,  through  the  high-level  tech¬ 
niques  for  combining  data  from  multiple  sensors 
and  different  sensing  modalities.  Presentations  pri¬ 
marily  focus  on  developments  in  computer  algo¬ 
rithms  for  sensory  data  processing  and  representa¬ 
tion  of  an  unmanned  underwater  vehicle's  environ¬ 
ment,  and  include:  perceptual  strategies;  physics- 
based  sensing  and  world  modeling;  techniques  for 
scene  reconstruction;  merging  data  over  extended 
periods  of  time;  target  acquisition,  recognition,  and 
classification;  and  the  trade-offs  between  global 
modeling  and  task-relative  perception. 

Autonomus  obstacle  avoidance  using  visual  fixation 
and  looming 

93N23442 

JOARDER,  KUNAL;  RAVIV,  DANIEL  Sponsored 
by  National  Inst,  of  Standards  and  Technology, 
Gaithersburg,  MD  Avail:  CASI  HC  A03/MF  A01 
The  paper  describes  a  vision-based  method  for 
avoiding  obstacles  using  the  concepts  of  visual 
looming  and  fixating  motion.  Visual  looming  refers 
to  the  expansion  of  images  of  objects  in  the  retina. 
Usually,  this  is  due  to  the  decreasing  distance  be¬ 
tween  the  observer  and  the  object.  An  increasing 
looming  value  signifies  an  increasing  threat  of  colli¬ 
sion  with  the  object.  The  visual  task  of  avoiding 
collision  can  be  further  simplified  by  purposive 
control  of  visual  fixation  at  the  objects  in  front  of 
the  moving  camera.  Using  these  two  basic  concepts, 
real  time  obstacle  avoidance  in  a  tight  perception- 
action  loop  is  implemented.  Three-dimensional 
space  in  front  of  the  camera  is  divided  into  zones 
with  various  degrees  of  looming-based  threat  of 
collision.  For  each  obstacle  seen  by  a  fixating  cam¬ 
era,  looming  and  its  time  derivative  are  calculated 
directly  from  the  2D  image.  Depending  on  the 
threat  posed  by  an  obstacle,  a  course  change  is 
dictated.  This  looming  based  approach  is  simple, 
independent  of  the  size  of  the  3D  object  and  its 
range,  and  involves  simple  quantitative  measure¬ 
ments.  Results  pertinent  to  a  camera  on  a  robot  arm 
navigating  between  obstacles  are  presented. 

Sensor-based  whole-arm  obstacle  avoidance 
utilizing  ASIC  technology 

93N22819 

WINTENBERG,  A.  L.;  ERICSON,  M.  N.;  BAB¬ 
COCK,  S.  M.;  ARMSTRONG,  G.  A.;  BRITTON, 

C.  L.,  JR.;  BUTLER,  P.  L.;  HAMEL,  W.  R.;  NEW¬ 
PORT,  D.  F.  (Tennessee  Univ.,  Knoxville.)  Present¬ 
ed  at  the  5th  Topical  Meeting  on  Robotics  and 
Remote  Systems,  Knoxville,  TN,  26-29  Apr.  1993 
Avail:  CASI  HC  A03/MF  A01 
Operation  of  manipulator  systems  in  poorly  defined 
work  environments  often  presents  a  significant  haz¬ 
ard  to  both  the  robotic  assembly  and  the  environ¬ 
ment.  In  applications  relating  to  the  Environmental 
Restoration  and  Waste  Management  (ER&WM) 
Program,  many  of  the  environments  are  considered 
hazardous,  both,  in  the  structure  and  composition  of 
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the  environment.  Use  of  a  sensing  system  that  pro¬ 
vides  information  to  the  manipulator  control  unit 
regarding  obstacles  in  close  proximity  will  provide 
protection  against  collisions.  A  hierarchical  design 
and  implementation  of  a  whole-arm  obstacle  avoid¬ 
ance  system  is  presented.  The  system  is  based  on 
capacitive  sensors  configured  as  bracelets  for  prox¬ 
imity  sensing.  Each  bracelet  contains  a  number  of 
sensor  nodes  and  a  processor  for  sensor  node  con¬ 
trol  and  readout,  and  communications  with  a  higher 
level  host,  common  to  all  bracelets.  The  host  con¬ 
trols  the  entire  sensing  network  and  communicates 
proximity  information  to  the  manipulator  controller. 
The  overall  architecture  of  this  system  is  discussed 
with  detail  on  the  individual  system  modules.  De¬ 
tails  of  an  application  specific  integrated  circuit 
(ASIC)  designed  to  implement  the  sensor  node 
electronics  are  presented.  Justifications  for  the  gen¬ 
eral  measurement  methods  and  associated  imple¬ 
mentation  are  discussed.  Additionally,  the  current 
state  of  development  including  measured  data  is 
presented. 

Advanced  telerobotic  control  using  neural  networks 

93N22364 

PAP,  ROBERT  M.;  ATKINS,  MARK;  COX, 
CHADWICK;  GLOVER,  CHARLES;  KISSEL, 
RALPH;  SAEKS,  RICHARD  (Accurate  Automation 
Corp.,  Chattanooga,  TN.);  (Accurate  Automation 
Corp.,  Chattanooga,  TN.);  (Accurate  Automation 
Corp.,  Chattanooga,  TN.);  (Tennessee  State  Univ., 
Nashville.);  (Accurate  Automation  Corp.,  Chatta¬ 
nooga,  TN.)  In  NASA.  Johnson  Space  Center,  Pro¬ 
ceedings  of  the  Third  International  Workshop  on 
Neural  Networks  and  Fuzzy  Logic,  Volume  1  p 
93-94  (SEE  N93-22351  08-63)  Sponsored  in  part  by 
ONR  and  NSF  Avail:  CASI  HC  A01/MF  A03 
Accurate  Automation  is  designing  and  developing 
adaptive  decentralized  joint  controllers  using  neural 
networks.  We  are  then  implementing  these  in  hard¬ 
ware  for  the  Marshall  Space  Flight  Center  PFMA  as 
well  as  to  be  usable  for  the  Remote  Manipulator 
System  (RMS)  robot  arm.  Our  design  is  being  real¬ 
ized  in  hardware  after  completion  of  the  software 
simulation.  This  is  implemented  using  a  Functional- 
Link  neural  network. 

A  fault-tolerant  intelligent  robotic  control  system 

93N22159 

MARZWELL,  NEVILLE  I.;  TSO,  KAM  SING 
(SoHaR,  Inc.,  Beverly  Hills,  CA.)  In  NASA,  Wash¬ 
ington,  Technology  2002:  The  Third  National  Tech¬ 
nology  Transfer  Conference  and  Exposition,  Vol¬ 
ume  2  p  101-110  (SEE  N93-22149  08-99)  Avail: 
CASI  HC  A02/MF  A04 

This  paper  describes  the  concept,  design,  and  fea¬ 
tures  of  a  fault-tolerant  intelligent  robotic  control 
system  being  developed  for  space  and  commercial 
applications  that  require  high  dependability.  The 
comprehensive  strategy  integrates  system  level  hard¬ 
ware/software  fault  tolerance  with  task  level  han¬ 
dling  of  uncertainties  and  unexpected  events  for 
robotic  control.  The  underlying  architecture  for 


system  level  fault  tolerance  is  the  distributed  recov¬ 
ery  block  which  protects  against  application  soft¬ 
ware,  system  software,  hardware,  and  network  fail¬ 
ures.  Task  level  fault  tolerance  provisions  are  im¬ 
plemented  in  a  knowledge-based  system  which 
utilizes  advanced  automation  techniques  such  as 
rule-based  and  model-based  reasoning  to  monitor, 
diagnose,  and  recover  from  unexpected  events.  The 
two  level  design  provides  tolerance  of  two  or  more 
faults  occurring  serially  at  any  level  of  command, 
control,  sensing,  or  actuation.  The  potential  benefits 
of  such  a  fault  tolerant  robotic  control  system  in¬ 
clude:  (1)  a  minimized  potential  for  damage  to 
humans,  the  work  site,  and  the  robot  itself;  (2)  con¬ 
tinuous  operation  with  a  minimum  of  uncommanded 
motion  in  the  presence  of  failures;  and  (3)  more 
reliable  autonomous  operation  providing  increased 
efficiency  in  the  execution  of  robotic  tasks  and 
decreased  demand  on  human  operators  for  control¬ 
ling  and  monitoring  the  robotic  servicing  routines. 

Avoiding  space  robot  collisions  utilizing  the 
NASGSFC  tri-mode  skin  sensor 

93N21812 

PRINZ,  F.  B.  S.;  MAHALINGAM,  S.  Avail:  CASI 
HC  A03/MF  A01 

A  capacitance  based  proximity  sensor,  the 
'Capaciflector'  (Vranish  92),  has  been  developed  at 
the  Goddard  Space  Flight  Center  of  NASA.  We  had 
investigated  the  use  of  this  sensor  for  avoiding  and 
maneuvering  around  unexpected  objects 
(Mahalingam  92).  The  approach  developed  there 
would  help  in  executing  collision-free  gross  mo¬ 
tions.  Another  important  aspect  of  robot  motion 
planning  is  fine  motion  planning.  Let  us  classify 
manipulator  robot  motion  planning  into  two  groups 
at  the  task  level:  gross  motion  planning  and  fine 
motion  planning.  We  use  the  term  'gross  planning' 
where  the  major  degrees  of  freedom  of  the  robot 
execute  large  motions,  for  example,  the  motion  of  a 
robot  in  a  pick  and  place  type  operation.  We  use 
the  term  'fine  motion'  to  indicate  motions  of  the 
robot  where  the  large  dofs  do  not  move  much,  and 
move  far  less  than  the  mirror  dofs,  such  as  in  in¬ 
serting  a  peg  in  a  hole.  In  this  report  we  describe 
our  experiments  and  experiences  in  this  area. 

Reliable  fusion  of  control  and  sensing  in  intelligent 
machines 

93N21371 

MCINROY,  JOHN  E.  Avail:  CASI  HC  A08/ 

MF  A02 

Although  robotics  research  has  produced  a  wealth 
of  sophisticated  control  and  sensing  algorithms,  very 
little  research  has  been  aimed  at  reliably  combining 
these  control  and  sensing  strategies  so  that  a  specif¬ 
ic  task  can  be  executed.  To  improve  the  reliability 
of  robotic  systems,  analytic  techniques  are  devel¬ 
oped  for  calculating  the  probability  that  a  particular 
combination  of  control  and  sensing  algorithms  will 
satisfy  the  required  specifications.  The  probability 
can  then  be  used  to  assess  the  reliability  of  the 
design.  An  entropy  formulation  is  first  used  to 
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quickly  eliminate  designs  not  capable  of  meeting 
the  specifications.  Next,  a  framework  for  analyzing 
reliability  based  on  the  first  order  second  moment 
methods  of  structural  engineering  is  proposed.  To 
ensure  performance  over  an  interval  of  time,  lower 
bounds  on  the  reliability  of  meeting  a  set  of  qua¬ 
dratic  specifications  with  a  Gaussian  discrete  time 
invariant  control  system  are  derived.  A  case  study 
analyzing  visual  positioning  in  robotic  system  is 
considered.  The  reliability  of  meeting  timing  and 
positioning  specifications  in  the  presence  of  camera 
pixel  truncation,  forward  and  inverse  kinematic 
errors,  and  Gaussian  joint  measurement  noise  is 
determined.  This  information  is  used  to  select  a 
visual  sensing  strategy,  a  kinematic  algorithm,  and  a 
discrete  compensator  capable  of  accomplishing  the 
desired  task.  Simulation  results  using  PUMA  560 
kinematic  and  dynamic  characteristics  are  presented. 

Case-based  reasoning  for  real-time  problem  solving 

93N19865 

HAMMOND,  KRISTIAN;  OWENS,  CHRISTO¬ 
PHER;  MARTIN,  CHARLES  Avail:  CASI  HC 
A03/MF  A01 

This  document  summarizes  the  University  of  Chica¬ 
go  Artificial  Intelligence  laboratory’s  work  on  ap¬ 
plying  case-based  methods  to  intelligent  real-time 
problem  solving.  An  approach  to  problem  solving 
involving  the  storage,  retrieval,  adaptation,  and 
re-use  of  successful  strategies  is  outlined.  The  report 
describes  work  on  an  overall  control  architecture,  on 
methodological  issues  in  the  development  of  repre¬ 
sentation  vocabularies,  and  on  memory  organization 
for  efficient  storage  and  retrieval  of  cases.  Work  on 
six  projects  is  described  including  work  on  robot 
planning,  the  dynamic  repair  of  transportation 
schedules,  multi-agent  cooperative  planning, 
case-based  design,  and  active  perception.  The  result 
of  this  work  is  a  model  of  planning  and  execution 
that  handles  the  complexity  and  instability  of  a 
dynamic,  real-time  environment  by  making  use  of 
known  plans  stored  in  memory. 

Cooperation  of  mobile  robots  for  accident  scene 
inspection 

93N 19478 

BYRNE,  R.  H.;  HARRINGTON,  J.  Presented  at  the 
ISRAM  1992:  4th  International  Symposium  on 
Robotics  and  Manufacturing,  Sante  Fe,  NM,  11-13 
Nov.  1992  Avail:  CASI  HC  A02/MF  A01 
A  telerobotic  system  demonstration  was  developed 
for  the  Department  of  Energy's  Accident  Response 
group  to  highlight  the  applications  of  telerobotic 
vehicles  to  accident  site  inspection.  The 
proof-of-principle  system  employs  two  mobile  ro¬ 
bots,  Dixie  and  RAYBOT,  to  inspect  a  simulated 
accident  site.  Both  robots  are  controlled  serially 
from  a  single  driving  station,  allowing  an  operator 
to  take  advantage  of  having  multiple  robots  at  the 
scene.  The  telerobotic  system  is  described  and  some 
of  the  advantages  of  having  more  than  one  robot 
present  are  discussed.  Future  plans  for  the  system 
are  also  presented. 


Nonlinear  robust  controller  design  for  multi-robot 
systems  with  unknown  payloads 

93N19462 

SONG,  Y.  D.;  ANDERSON,  J.  N.;  HOMAIFAR, 

A.;  LAI,  H.  Y.  (Tennessee  Technological  Univ., 
Cookeville.)  In  its  The  Center  for  Aerospace  Re¬ 
search:  A  NASA  Center  of  Excellence  at  North 
Carolina  Agricultural  and  Technical  State  University 
16  p  (SEE  N93-19452  06-80)  Avail:  CASI  HC 
A03/MF  A03 

This  work  is  concerned  with  the  control  problem  of 
a  multi-robot  system  handling  a  payload  with  un¬ 
known  mass  properties.  Force  constraints  at  the 
grasp  points  are  considered.  Robust  control  schemes 
are  proposed  that  cope  with  the  model  uncertainty 
and  achieve  asymptotic  path  tracking.  To  deal  with 
the  force  constraints,  a  strategy  for  optimally  shar¬ 
ing  the  task  is  suggested.  This  strategy  basically 
consists  of  two  steps.  The  first  detects  the  robots 
that  need  help  and  the  second  arranges  that  help.  It 
is  shown  that  the  overall  system  is  not  only  robust 
to  uncertain  payload  parameters,  but  also  satisfies 
the  force  constraints. 

A  safety-based  decision  making  architecture  for 
autonomous  systems 

93N19362 

MUSTO,  JOSEPH  C.;  LAUDERBAUGH,  L.  K. 
Avail:  CASI  HC  A03/MF  A01 
Engineering  systems  designed  specifically  for  space 
applications  often  exhibit  a  high  level  of  autonomy 
in  the  control  and  decision-making  architecture.  As 
the  level  of  autonomy  increases,  more  emphasis 
must  be  placed  on  assimilating  the  safety  functions 
normally  executed  at  the  hardware  level  or  by  hu¬ 
man  supervisors  into  the  control  architecture  of  the 
system.  The  development  of  a  decision-making 
structure  which  utilizes  information  on  system  safe¬ 
ty  is  detailed.  A  quantitative  measure  of  system 
safety,  called  the  safety  self-information,  is  defined. 
This  measure  is  analogous  to  the  reliability  self¬ 
information  defined  by  Mclnroy  and  Saridis,  but 
includes  weighting  of  task  constraints  to  provide  a 
measure  of  both  reliability  and  cost.  An  example  is 
presented  in  which  the  safety  self-information  is 
used  as  a  decision  criterion  in  a  mobile  robot  con¬ 
troller.  The  safety  self-information  is  shown  to  be 
consistent  with  the  entropy-based  Theory  of  Intelli¬ 
gent  Machines  defined  by  Saridis. 

Teleprogramming:  Remote  site  robot  task  execution 

93N 19046 

LINDSAY,  THOMAS  STEWART  Avail:  Univ. 
Microfilms  Order  No.  DA9235172 
This  dissertation  describes  a  remote  site  robot 
workcell  for  teleoperation  with  communication 
delays  on  the  order  of  20  seconds.  In  these  situa¬ 
tions,  direct  teleoperation  becomes  impossible  due 
to  the  delays  in  visual  and  force  feedback. 
Teleprogramming  has  been  developed  in  order  to 
overcome  this  problem.  An  integral  part  of 
teleprogramming  is  a  semi-autonomous  remote  site 
robot  system.  The  remote  system  is  composed  of  a 
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robot  manipulator,  sensors,  controlling  computer, 
and  manipulator  tools.  The  constraints  on  the  re¬ 
mote  site  system  and  the  amount  of  autonomy  need¬ 
ed  are  defined  partially  by  the  teleprogramming 
system  and  partially  by  the  needs  of  the  remote 
system.  Development  of  the  remote  site  system  for 
teleprogramming  evokes  some  pertinent  research 
issues:  low  level  manipulator  control,  semi-autono¬ 
mous  command  execution,  and  remote  site  tool 
usage.  Low  level  manipulator  control  is  based  on  a 
hybrid  control  scheme  using  wrist-based  sensory 
feedback.  Implementation  of  this  control  is  present¬ 
ed  and  problems  related  to  controlling  the  manipula¬ 
tor  in  an  arbitrary  frame  are  investigated.  High  level 
commands  are  executed  at  the  remote  site  in  small 
autonomous  steps.  Implementation  of  tolerance 
checks  and  guarded  moves  are  presented,  including 
error  detection  and  the  detection  of  motion  termina¬ 
tion  conditions  in  a  partially  known  environment. 
Power  tools  introduce  redundant  degrees  of  freedom 
into  the  manipulator/tool  system.  To  control  this 
redundant  system,  the  tool  is  actively  controlled  in 
its  natural  degree  of  freedom  and  the  corresponding 
degree  of  freedom  in  the  manipulator  becomes  pas¬ 
sive.  Feedback  for  the  manipulator/tool  system  is 
supplied  by  the  wrist-based  sensor.  Two  examples 
of  sensing  and  control  for  tools  are  presented.  This 
research  has  resulted  in  the  development  of  a  re¬ 
mote  site  system  for  teleprogramming.  The  remote 
system,  however,  is  both  time-delay  and  input  inde¬ 
pendent.  The  characteristics  of  the  system,  including 
the  compliance,  flexibility,  and  semi-autonomity,  are 
useful  to  a  wide  variety  of  robotics  applications, 
including  manufacturing  and  direct  teleoperation. 

Voice  control  of  a  dual-arm  telerobot 

93N 19044 

HABERLEIN,  ROBERT  ARTHUR  Avail:  Univ. 
Microfilms  Order  No.  DA9238647 
This  investigation  explores  voice  control  of  a 
dual-ann  telerobot.  A  literature  review  of  voice 
control,  voice  technology  and  work  measurements  is 
conducted.  This  review  includes  a  discussion  of 
important  voice  technology  topics,  a  survey  of  com¬ 
mercial  voice  equipment,  and  a  study  of  industrial 
and  vocational  work  measurement  techniques.  A 
voice  control  system  is  created  for  two  Kraft  GRIPS 
Master-Slave  telerobotic  manipulators.  This  system 
is  based  upon  the  concept  of  distributed  computer 
control  using  inexpensive  PC- AT  computers  that 
exchange  information  according  to  special  commu¬ 
nication  and  command  protocols.  The  voice  control 
system  consists  of  four  separate  sub-systems;  a 
Camera  Sub-system  that  controls  a  motorized  cam¬ 
era  mount,  a  Teach  Pendant  Sub-system  that  emu¬ 
lates  two  standard  Termiflex  teach  pendants,  a 
Switch  Sub-system  that  controls  the  Kraft  Master 
switches,  and  a  Voice  Sub-system  that  accepts  the 
operator's  vocal  commands  and  broadcasts 
digitally-recorded  messages.  The  Voice  Sub-system 
utilizes  a  Votan  VPC-2100  recognition  board  and  a 
Tl-Speech  synthesis  board.  The  vocal  commands 
are  organized  into  a  hierarchical  structure  based 


upon  the  fire-and-forget  control  scheme.  A  visual 
display  of  the  vocal  command  status  is  also  de¬ 
tailed.  In  order  to  measure  the  effect  of  the  voice 
control  system  upon  the  work  performance  of  the 
telerobot,  a  formal  experimental  plan  is  described 
using  twenty-four  untrained  operators  divided  into  a 
voice  group  and  a  control  group.  Each  group  per¬ 
forms  an  experimental  taskset  using  modified 
peg-in-hole  vocational  rehabilitation  assessment  test 
equipment.  The  experimental  taskset  consists  of 
eight  separate  subtasks  that  exercise  each  of  the 
four  voice  control  sub-systems.  The  times  to  com¬ 
plete  the  subtasks  are  recorded  to  score  each  group's 
work  performance.  A  split-plot  ANOVA  of  the 
performance  scores  reveals  significant  group  im¬ 
provements  in  both  the  mean  performance  and  the 
performance  variance  for  those  tasks  which  involve 
control  of  the  telerobot  and  its  peripheral  systems. 
No  significant  group  differences  are  found  for  those 
subtasks  which  chiefly  involve  the  dexterity  of  the 
telerobotic  Slaves. 

ORCCAD:  Towards  an  open  robot  controller 
computer  aided  design  system 

93N18519 

JOUBERT,  ANTOINE;  SIMON,  DANIEL  (Ecole 
Nationale  Superieure  des  Mines,  Valbonne,  France  ) 
Sponsored  in  part  by  Commission  of  the  European 
Communities  Avail:  CASI  HC  A03/MF  A01 
The  implementation  of  robotics  tasks  via  various 
robot  controller  architectures  using  the  ORCCAD 
system  is  addressed.  The  development  of  new 
robotic  tasks  deals  with  many  different  techniques 
and  scientific  fields.  Automatic  control  is  involved 
in  control  laws  analysis  while  computer  science 
tools  are  used  to  produce  efficient  real  time  pro¬ 
grams.  Following  the  development  of  a  full  exam¬ 
ple,  a  methodology  for  the  design  of  new  robotics 
tasks  and  a  review  of  the  tools  to  be  used  at  each 
step  of  the  task  development,  are  given. 

Composite  video  and  graphics  display  for  camera 
viewing  systems  in  robotics  and  teleopcration 

93N18284 

DINER,  DANIEL  B.;  VENEMA,  STEVEN  C.  (Jet 
Propulsion  Lab.,  California  Inst,  of  Tech.,  Pasade¬ 
na.);  (Jet  Propulsion  Lab.,  California  Inst,  of  Tech., 
Pasadena.)  Filed  17  Jun.  1991  Supersedes 
N92-10126  (30  -  1,  p  25)  Avail:  US  Patent  and 
Trademark  Office 

A  system  for  real-time  video  image  display  for 
robotics  or  remote-vehicle  teleoperation  is  described 
that  has  at  least  one  robot  arm  or  remotely  operated 
vehicle  controlled  by  an  operator  through 
hand-controllers,  and  one  or  more  television  camer¬ 
as  and  optional  lighting  element.  The  system  has  at 
least  one  television  monitor  for  display  of  a  televi¬ 
sion  image  from  a  selected  camera  and  the  ability  to 
select  one  of  the  cameras  for  image  display.  Graph¬ 
ics  are  generated  with  icons  of  cameras  and  lighting 
elements  for  display  surrounding  the  television 
image  to  provide  the  operator  information  on:  the 
location  and  orientation  of  each  camera  and  lighting 
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element;  the  region  of  illumination  of  each  lighting 
element;  the  viewed  region  and  range  of  focus  of 
each  camera;  which  camera  is  currently  selected  for 
image  display  for  each  monitor;  and  when  the  con¬ 
troller  coordinate  for  said  robot  arms  or  remotely 
operated  vehicles  have  been  transformed  to  corre¬ 
spond  to  coordinates  of  a  selected  or  nonselected 
camera. 

Design  requirements  for  force  reflecting  master 
controllers 

93N18035 

SRINIVASSEN,  M.  Avail:  CASI  HC  A03/MF  A01 
The  design  criteria  imposed  by  the  capabilities  of 
the  human  user  on  the  design  of  force  reflecting 
controllers  for  hands  and  arms  are  discussed.  This 
paper  contains  four  sections.  First,  we  present  a 
framework  of  questions  regarding  human  capabili¬ 
ties.  Second,  a  subset  of  the  criteria  is  selected  as 
the  critical  task  set.  Third,  values  for  this  task  set 
are  either  presented  or  engineering  experiments  for 
determining  the  values  are  given.  Lastly,  the  rela¬ 
tionship  between  the  critical  task  set  and  the  engi¬ 
neering  specifications  for  the  machine  are  given. 

The  framework  discusses  in  a  broad  way  all  of  the 
kinesthetic,  kinematic,  and  tactile  capabilities  of  the 
human  hand  and  arm.  A  machine  which  met  or 
exceeded  all  of  these  capabilities  would  present  a 
true  virtual  reality  interface.  Many  of  these  criteria 
cannot  be  met  with  current  technologies.  Therefore, 
the  critical  sensing/actuation  dimensions  for  per¬ 
forming  a  set  of  tasks  on  a  task  board  must  be  de¬ 
termined.  We  present  a  hypothesized  set  of  criteria 
based  on  experience  and  the  literature,  and  discuss 
some  simple  experiments  that  can  be  used  to  clarify 
the  selection.  For  all  of  the  capabilities,  we  present 
values  that  are  available  in  the  literature.  The  source 
of  each  value  is  referenced,  and  the  experiment  is 
briefly  summarized  and  critiqued.  For  critical  capa¬ 
bilities  with  unknown  values,  we  have  designed 
some  engineering  experiments  to  determine  the 
values.  In  the  design  of  the  machine,  only  a  subset 
of  the  human  capability  criteria  are  important  in 
determining  engineering  design  specifications.  For 
instance,  force  threshold  without  any  bias  load  de¬ 
termines  the  maximum  reflected  force  that  the 
mechanism  can  apply  in  free  motion  mode.  This  in 
turn  determines  the  friction  that  can  be  tolerated  in 
the  design.  We  present  the  important  engineering 
design  parameters,  and  show  how  these  can  be 
determined  from  the  human  capabilities. 

Automatic  tuning  for  a  teleoperated  arm  controller 

93N17481 

KRESS,  R.  L.;  JANSEN,  J.  F.  Presented  at  the  31st 
IEEE  Conference  on  Decision  and  Control,  Tuscon, 
AZ,  16-18  Dec.  1992  Avail:  CASI  HC  A01/ 

MF  A01 

This  paper  addresses  the  problem  of  determining  an 
optimal  set  of  gains  for  a  controller  for  a 
teleoperated  arm.  Specifically,  an  automatic  tuning 
technique  was  applied  and  investigated  for  tuning 
an  independent-joint  proportional-derivative  con¬ 


troller  for  a  teleoperated  manipulator.  The  Hooke 
and  Jeeves  method  is  used  in  conjunction  with  a 
one-dimensional  search  routine  in  the  tuning  algo¬ 
rithm.  The  algorithm  was  used  to  optimize  gains  for 
a  two-link  teleoperator  simulation  and  the  results  of 
several  optimizations  were  used  to  determine  the 
best  form  for  an  input  trajectory  and  cost  function. 
The  desired  joint  angle  trajectory  is  taken  from 
low-pass  filtered  step  inputs  with  randomly  generat¬ 
ed  magnitudes,  which  vary  at  a  predetermined  inter¬ 
val.  Both  positive  and  negative  angles  are  generat¬ 
ed,  but  they  are  constrained  to  lie  within  the  manip¬ 
ulator  work  space.  It  was  determined  that  the  cost 
function  should  be  based  on  tracking  error,  peak 
position  error  over  the  entire  desired  path,  over¬ 
shoot,  actuator  torque  bounds,  and  gain  limits.  The 
optimized  gains  obtained  from  the  simulation  were 
applied  to  an  actual  teleoperator  and  some  improve¬ 
ment  was  seen. 

Implementation  of  robotic  force  control  with 
position  accommodation 

93N16663 

RYAN,  MICHAEL  J.  Avail:  CASI  HC  A05/ 

MF  A02 

As  the  need  for  robotic  manipulation  in  fields  such 
as  manufacturing  and  telerobotics  increases,  so  does 
the  need  for  effective  methods  of  controlling  the 
interaction  forces  between  the  manipulators  and 
their  environment.  Position  Accommodation  (PA)  is 
a  form  of  robotic  force  control  where  the  nominal 
path  of  the  manipulator  is  modified  in  response  to 
forces  and  torques  sensed  at  the  tool-tip  of  the  ma¬ 
nipulator.  The  response  is  tailored  such  that  the 
manipulator  emulates  a  mechanical  impedance  to  its 
environment.  PA  falls  under  the  category  of 
position-based  robotic  force  control,  and  may  be 
viewed  as  a  form  of  Impedance  Control.  The  prac¬ 
tical  implementations  are  explored  of  PA  into  an  18 
degree-of-freedom  robotic  testbed  consisting  of  two 
PUMA  560  arms  mounted  on  two  3  DOF  position¬ 
ing  platforms.  Single  and  dual-arm  architectures  for 
PA  are  presented  along  with  some  experimental 
results.  Characteristics  of  position-based  force  con¬ 
trol  are  discussed,  along  with  some  of  the  limita¬ 
tions  of  PA. 

A  telerobotic  digital  controller  system 

93N16658 

BROWN,  RICHARD  J.  Avail:  CASI  HC  A08/ 

MF  A02 

This  system  is  a  network  of  joint  mounted  dual  axes 
digital  servo-controllers  (DDSC),  providing  control 
of  various  joints  and  end  effectors  of  different 
robotic  systems.  This  report  provides  description  of 
and  user  required  information  for  the  Digital  Con¬ 
troller  System  Network  (DSCN)  and,  in  particular, 
the  DDSC,  Model  DDSC-2,  developed  to  perform 
the  controller  functions.  The  DDSC  can  control  3 
phase  brushless  or  brush  type  DC  motors,  requiring 
up  to  8  amps.  Only  four  wires,  two  for  power  and  2 
for  serial  communication,  are  required,  except  for 
local  sensor  and  motor  connections.  This  highly 
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capable,  very  flexible,  programmable  servo-control¬ 
ler,  contained  on  a  single,  compact  printed  circuit 
board  measuring  only  4.5  x  5.1  inches,  is  applicable 
to  control  systems  of  all  types  from  sub-arc  second 
precision  pointing  to  control  of  robotic  joints  and 
end  effectors.  This  document  concentrates  on  the 
robotic  applications  for  the  DDSC. 

Telerobotic  control  of  a  mobile  coordinated  robotic 

server 

93N16387 

LEE,  GORDON  Avail:  CAS1  HC  A05/MF  A02 
Results  from  the  Master's  Degree  Thesis  of  Mr. 
Robert  Stanley,  a  graduate  student  supervised  by  the 
principal  investigator  on  this  project  is  reported.  The 
goal  of  this  effort  is  to  develop  advanced  control 
methods  for  flexible  space  manipulator  systems.  As 
such,  a  fuzzy  logic  controller  has  been  developed  in 
which  model  structure  as  well  as  parameter  con¬ 
straints  are  not  required  for  compensation.  A  gener¬ 
al  rule  base  is  formulated  using  quantized  linguistic 
terms;  it  is  then  augmented  to  a  traditional  integral 
control.  The  resulting  hybrid  fuzzy  controller  stabi¬ 
lizes  the  structure  over  a  broad  range  of  uncertain¬ 
ties,  including  unknown  initial  conditions.  An 
off-line  tuning  approach  using  phase  portraits  gives 
further  insight  into  the  algorithm.  The  approach  was 
applied  to  a  three-degree-of-freedom  manipulator 
system  -  the  prototype  of  the  coordinated  flexible 
manipulator  system  currently  being  designed  and 
built  at  North  Carolina  State  University. 

Stereo  vision  controlled  bilateral  telerobotic  remote 
assembly  station 

93N16120 

DEWITT,  ROBERT  L.  Avail:  CASI  HC  A04/ 

MF  A01 

The  objective  of  this  project  was  to  develop  a  bilat¬ 
eral  six  degree-of-freedom  telerobotic  component 
assembly  station  utilizing  remote  stereo  vision  as¬ 
sisted  control.  The  component  assembly  station 
consists  of  two  Unimation  Puma  260  robot  arms 
and  their  associated  controls,  two  Panasonic  minia¬ 
ture  camera  systems,  and  an  air  compressor.  The 
operator  controls  the  assembly  station  remotely  via 
kinematically  similar  master  controllers.  A  Zenith 
386  personal  computer  acts  as  an  interface  and 
system  control  between  the  human  operator's  con¬ 
trols  and  the  Val  II  computer  controlling  the  arms. 

A  series  of  tasks,  ranging  in  complexity  and  diffi¬ 
culty,  was  utilized  to  assess  and  demonstrate  the 
performance  of  the  complete  system. 

Knowledge-based  planning 

93N15812 

MCDERMOTT,  DREW;  HAGER,  GREGORY 
Avail:  CASI  HC  A02/MF  A01 
The  goal  of  our  project  is  to  study  planning  for 
autonomous  agents  with  imperfect  sensors  in  a 
dynamic  world.  Such  agents  must  confront  several 
problems:  (1)  how  to  synchronize  plan  execution 
with  plan  refinement;  (2)  how  to  generate  reason¬ 
able  plans  quickly  for  complex  goals,  and  improve 


them  later;  (3)  how  to  trade  off  sensor-processing 
time  against  the  quality  of  information;  and  (4)  how 
to  learn  the  stmcture  of  the  environment  as  plan 
execution  proceeds. 

SMART:  A  modular  architecture  for  robotics  and 
teleoperation 

93N15385 

ANDERSON,  R.  J.  Presented  at  the  ISRAM  1992: 
Fourth  International  Symposium  on  Robotics  and 
Manufacturing,  Santa  Fe,  NM,  11-13  Nov.  1992 
Avail:  CASI  HC  A02/MF  A01 
This  paper  introduces  SMART:  Sandia  National 
Laboratory’s  Modular  Architecture  for  Robotics  and 
Teleoperation.  SMART  is  designed  to  integrate  the 
different  slave  devices  (e.g.,  large  hydraulic  arms, 
mobile  manipulators,  and  gantry  robots),  sensors 
(e.g.,  ultra-sonic  sensors  and  force  sensors),  and 
input  devices,  (e.g.,  track  ball,  force-reflecting  mas¬ 
ter,  and  autonomous  trajectory  generators)  required 
for  waste  management  and  environmental  restora¬ 
tion  tasks.  The  modular  architecture  allows  for  rapid 
synthesis  of  complex  telerobotic  systems.  This  pa¬ 
per  introduces  some  sample  modules  and  illustrates 
how  the  modules  can  be  connected  to  achieve 
telerobotic  behaviors.  Examples  include  autonomous 
control,  impedance  control,  and  enhanced  bilateral 
teleoperation. 

Adaptive  control  of  a  manipulator  in  cartesian  space 

93N15308 

MURTY,  VENKATAESH  V.;  SHUNMUGAM,  M. 
S.  In  CASA-RI/SME,  Proceedings  of  the  Interna¬ 
tional  Conference  on  CACAM  and  AMT.  Volume 
3:  IFIP/Robotics:  International  Sessions  on  Robots 
and  Assembly  5  p  (SEE  N93-15298  04-61)  Avail: 
CASI  HC  A01/MF  A02 

A  cartesian  based  nonlinear  adaptive  controller  is 
presented  for  the  trajectory  control  of  a  robot  ma¬ 
nipulator.  The  stability  of  the  control  formulation  is 
proved  using  Lyapunov's  second  method.  The  un¬ 
known  parameters  of  the  system,  as  well  as  the 
variable  payload  are  estimated  on-line,  tending  the 
system  errors  to  zero.  This  formulation,  unlike  other 
adaptive  controllers  present  in  the  literature,  neither 
makes  any  ad  hoc  linearization  of  the  nonlinear 
equations  nor  requires  the  inversion  of  the  inertia 
matrix.  It  also  does  not  require  the  measurement  of 
acceleration  and  the  conversion  of  path  planning 
specifications  from  cartesian  space  to  joint  space. 
Trajectory  tracking,  with  unknown  parameter  values 
and  variable  payload,  is  simulated  to  show  the 
effectiveness  of  the  proposed  adaptive  control 
algorithm. 

Coordinated  motion  control  of  a  two-hand  assembly 

system 

93N15301 

BIDAUD,  P.;  FONTAINE,  D.;  BOUDIN,  F.; 
GUINOT,  J.  C.  In  CASA-RI/SME,  Proceedings  of 
the  International  Conference  on  CACAM  and  AMT. 
Volume  3:  IFIP/Robotics:  International  Sessions  on 
Robots  and  Assembly  4  p  (SEE  N93-15298  04-61) 
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Avail:  CAST  HC  A01/MF  A02 
This  paper  presents  the  implementation  of  an  origi¬ 
nal  cooperating  system  for  precise  assembly  opera¬ 
tions.  The  whole  mechanical  architecture  consists  of 
a  conventional  manipulator  robot  and  a 
micro-manipulator  setting  in  parallel.  The 
micro-manipulator  used  is  a  manipulator-gripper  in 
a  left-hand  configuration.  The  mechanism  of  the 
manipulator-gripper  was  designed  to  perform  active 
or  passive  fine  compliant  motions.  A  hybrid 
force/position  control  structure  is  used  to  control  the 
grasped  object  motions  in  operational  space.  A 
real-time  distributed  computer  system  has  been 
developed  for  coordinated-motion  control.  For  com¬ 
plex  assembly  tasks,  automatic  fine  motion  strate¬ 
gies  are  obtained  by  a  rule-based  expert  system. 

Methodology  for  robot  capability  and  performance 
characterization:  The  basis  for  robot  task  planning 

93N15300 

RUBINOVITZ,  JACOB;  WYSK,  RICHARD  A. 
(Pennsylvania  State  Univ.,  University  Park.)  In 
CASA-RI/SME,  Proceedings  of  the  International 
Conference  on  CACAM  and  AMT.  Volume  3: 
IFIP/Robotics:  International  Sessions  on  Robots  and 
Assembly  9  p  (SEE  N93-15298  04-61)  Avail: 

CASI  HC  A02/MF  A02 

The  motion  capabilities  and  limitations  of  ASAE's 
IRL6  welding  robot  were  studied,  including  timing 
of  its  motion  along  nine  different  straight  line  seg¬ 
ments,  ranging  in  length  from  10-300  mm.  Data 
collection  and  analysis  were  performed  automati¬ 
cally  by  a  personal  computer  linked  to  the  robot. 
Indirect  time  and  motion  study  was  performed  by 
measuring  accelerations  at  the  tool  center  point.  The 
programmed  motion  velocities  ranged  from  100- 
1 100  mm/sec.  A  model  for  determining  pro¬ 
grammed  robot  velocities  and  for  estimating  times 
of  motion,  while  taking  full  advantage  of  the  robot's 
capabilities  was  developed.  The  model  and  the  data 
collected  during  the  measurements  were  integrated 
into  a  database  of  robot  capabilities  which  was 
implemented  in  an  off-line  task-planning  and  pro¬ 
gramming  system  for  robotic  welding.  Alternative 
motion  paths  between  seams  were  evaluated,  and 
motion  velocities  were  assigned  to  these  paths. 
Similar  methodology  can  be  applied  to  different 
robots  by  implementing  the  established  database  for 
automated  task  planning. 

CAD-based  off-line  programming  and  sensor  signal 
processing  in  robotics 

93N15299 

SIEGLER,  ANDRAS;  BATHOR,  MIKLOS  In 
CASA-RI/SME,  Proceedings  of  the  International 
Conference  on  CACAM  and  AMT.  Volume  3: 
IFIP/Robotics:  International  Sessions  on  Robots  and 
Assembly  8  p  (SEE  N93-15298  04-61)  Avail: 

CASI  HC  A02/MF  A02 

The  first  part  of  the  paper  describes  a  three-dimen¬ 
sional  computer  animation  system  to  be  used  for 
creating  motion  programs  for  a  set  of  actors.  The 
animation  package  includes  a  language  for  describ¬ 


ing  spatial  motion  of  bodies  as  well  as  an  interac¬ 
tive  tool  to  be  used  for  assigning  geometrical  attrib¬ 
utes  to  them  and  for  guiding  and  visualizing  the 
play-back  of  motion  programs.  One  of  the  main 
uses  of  the  package  is  robot  programming  and  mo¬ 
tion  simulation.  A  geometrical  modeller,  which  is 
an  aid  in  creating  the  models  of  animated  bodies,  is 
also  described.  The  second  part  of  the  paper  gives 
an  overview  on  computer  vision  and  force  torque 
measurement  and  processing  units  developed  for 
robotics  and  other  automation  purposes. 

A  concept  for  a  supervised  autonomous  robot 

93N14950 

KALAYCIOGLU,  S.  In  Canadian  Space  Agency, 
Proceedings  of  the  Sixth  CASI  Conference  on  As¬ 
tronautics  17  p  (SEE  N93-14920  04-12)  Previously 
announced  in  IAA  as  A91-34956  Avail:  Issuing 
Activity  (Canadian  Aeronautics  and  Space  Inst.,  222 
Somerset  St.  W.,  Suite  601,  Ottawa,  Ontario  K2P 
OJ1  Canada) 

The  paper  describes  work  in  progress  at 
Thomson-CSF  Systems  Canada  Inc.  on  the  Mobile 
Servicing  System  (MSS)  Autonomous  Robotics 
Program.  The  main  objective  of  this  program  is  to 
define  and  plan  the  development  of  technologies 
required  to  provide  a  supervised  autonomous  opera¬ 
tion  capability  for  the  Special  Purpose  Dexterous 
Manipulator  (SPDM)  on  the  Mobile  Servicing  Sys¬ 
tem  (MSS).  In  this  paper,  a  telerobotics  system 
concept  is  introduced  and  a  summary  of  the  system 
requirements  is  given.  The  development  methodol¬ 
ogy  as  well  as  the  concept  for  a  supervised  autono¬ 
mous  robot  (telerobotics)  are  briefly  explained.  The 
functional  and  physical  architectures  of  the 
telerobotics  system  are  also  provided.  This  system 
will  be  responsible  for  carrying  out  operations  such 
as  assembly  and  maintenance  of  the  Space  Station 
Freedom;  loading  /  unloading  from  the  shuttle;  and 
retrieval  and  deployment  of  the  shuttle,  etc.  The 
paper  also  investigates  an  operational  scenario  for 
maintenance  of  the  Space  Station  Freedom  and 
briefly  describes  the  operational  scenario  for  chang¬ 
ing  an  orbital  replacement  unit  (ORU)  on  the  Mo¬ 
bile  Servicing  System.  The  functional  responsibili¬ 
ties  of  the  system  components  in  order  to  imple¬ 
ment  the  ORU  change  are  outlined. 

A  robotic  planning  system 

93N14949 

CHRYSTALL,  KEITH;  DAGNINO,  A.;  FEIGHAN, 
P.  In  Canadian  Space  Agency,  Proceedings  of  the 
Sixth  CASI  Conference  on  Astronautics  9  p  (SEE 
N93-14920  04-12)  Previously  announced  in  IAA  as 
A91-34955  Avail:  Issuing  Activity  (Canadian 
Aeronautics  and  Space  Inst.,  222  Somerset  St.  W., 
Suite  601,  Ottawa,  Ontario  K2P  OJ1  Canada) 

Space  station  robots  are  expected  to  be  viable  sub¬ 
stitutes  to  astronauts  in  the  execution  of  space  sta¬ 
tion  assembly,  repair  and  maintenance  tasks.  This 
paper  presents  a  multi-level  robotic  planning  system 
for  the  assembly  of  electronic  components  in  a 
printed  circuit  board,  employing  surface-mount 
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technology.  The  NASNBS  Standard  Reference 
Model  (NASREM)  control  architecture  was  used  as 
a  model  for  this  system.  The  NASREM  architecture 
is  hierarchically  structured  into  multiple  layers  of 
decreasing  complexity  and  horizontally  partitioned 
into  three  sections:  sensory  processing,  world  mod¬ 
elling,  and  task  decomposition.  The  approach  fol  ¬ 
lowed  in  this  project  was  to  develop  a  knowledge 
based  system  to  determine  the  activities  and  re¬ 
sources  required  to  assemble  components  in  a  print¬ 
ed  circuit  board.  Necessary  activities  are  then  de¬ 
composed  into  low  level  actions  that  can  be  execut¬ 
ed  by  robotic  workcells.  By  employing  this  system, 
robot  programming  is  simplified  as  assembly  tasks 
may  be  specified  very  concisely  and  all  the  required 
robot  actions  are  automatically  generated.  The  tech¬ 
nology  is  adaptable  to  both  space-based  and  terres¬ 
trial  robot  control  systems. 

Dynamics  and  control  of  a  rigid/flexible 
manipulator 

93N14684 

YEH,  CHUN-TIEN  Avail:  Univ.  Microfilms  Order 
No.  DA9229776 

Control  of  high-speed,  light-weight  robotic  manipu¬ 
lators  is  a  challenge  because  of  their  special  dynam¬ 
ic  characteristics.  A  two-stage  control  algorithm  for 
the  position  control  of  flexible  manipulators  is  pro¬ 
posed.  First,  the  more  complex,  flexible  robot  sys¬ 
tem  is  replaced  by  a  simplified  hypothetical  rigid 
body  system  (HRRA)  with  offline  trajectory  plan¬ 
ning.  This  reduces  the  complexity  of  the  controller 
design  for  the  flexible  robotic  arm.  A  parameter- 
optimization  approach  was  adopted  to  minimize  the 
difference  between  these  two  models  in  this  stage. 
Also,  a  comparison  of  computational  efficiency  is 
made  among  the  methods  of  calculus-of-variations, 
dynamic-programming,  and  the  proposed  parameter- 
optimization.  At  the  second  stage,  simple  linear 
state  feedback  controllers,  based  on  the  simplified 
hypothetical  rigid  body  model,  are  proposed  to 
control  the  actual  robotic  system.  With  the  feedback 
gains  selected  properly  by  the  pole-placement  and 
linear  quadratic  methods,  the  results  show  satisfac¬ 
tory  achievement  of  the  motion  objectives.  The 
algorithm  is  implemented  for  a  two-link  rigid/ 
flexible  robotic  arm,  and  the  results  indicate  that  the 
procedure  is  capable  of  providing  effective  control 
with  much  simpler  computational  requirements  than 
those  of  procedures  published  previously. 

Real-time  gaze  holding  in  binocular  robot  vision 

93N14582 

COOMBS,  DAVID  J.  Avail:  CASI  HC  A08/ 

MF  A02 

Using  a  binocular,  maneuverable  visual  system,  a 
robot  that  holds  its  gaze  on  a  visual  target  can  enjoy 
improved  visual  perception  and  performance  in 
interacting  with  the  world.  This  dissertation  exam¬ 
ines  the  problem  of  holding  gaze  on  a  moving  ob¬ 
ject  from  a  moving  platform,  without  requiring  the 
ability  to  recognize  the  target.  A  novel  aspect  of  the 
approach  taken  is  the  use  of  controlled  camera 


movements  to  simplify  the  visual  processing  neces¬ 
sary  to  keep  the  cameras  locked  on  the  target. 

A  gaze  holding  system  on  the  Rochester  robot's 
binocular  head  demonstrates  this  approach.  Even 
while  the  robot  is  moving,  the  cameras  are  able 
to  track  an  object  that  rotates  and  moves  in  three 
dimensions. 

Nonholonomic  motion  of  free-flying  space  robots 

93N13638 

MUKHERJEE,  RANJAN  Avail:  Univ.  Microfilms 
Order  No.  DA92 18468 

A  free-flying  space  vehicle  with  one  or  more  ma¬ 
nipulators  is  expected  to  perform  various  tasks  in 
space,  like  the  construction  of  space  stations,  de¬ 
ployment  of  large  antennas  and  solar  power  stations, 
and  the  servicing  and  maintenance  of  satellites.  The 
kinematics  and  dynamics  of  these  space  robots  are 
significantly  different  from  terrestrial  robots.  This  is 
because  of  the  nonholonomic  nature  of  the  momen¬ 
tum  constraints  that  govern  their  motion.  We  de¬ 
scribe  a  6-DOF  space  robot  by  nine  state  variables: 
the  six  joint  angles  of  the  manipulator  and  the  three 
dependent  Euler  angles  describing  the  orientation  of 
the  space  vehicle.  The  nonholonomy  in  the  mechan¬ 
ical  structure  manifests  itself  when  we  show  that  it 
is  possible  to  converge  all  the  nine  state  variables  to 
their  desired  values  by  controlling  only  the  six 
joints  of  the  manipulator.  We  plan  this  trajectory 
using  a  Liapunov  function  and  by  adopting  a  bi¬ 
directional  approach.  Another  interesting  feature  of 
free-flying  space  robots  is  the  presence  of  a  special 
kind  of  redundancy  in  its  mechanical  structure.  This 
redundancy,  which  we  term  as  nonholonomic  redun¬ 
dancy,  exhibits  itself  only  after  a  global  motion  and 
is  shown  to  exist  in  the  absence  of  ordinary  kine¬ 
matic  redundancy.  We  establish  methods  to  utilize 
nonholonomic  redundancy  for  avoiding  obstacles 
and  joint  limits,  while  planning  a  trajectory  for  the 
end-effector.  Finally  we  discuss  the  inverse  dynam¬ 
ics  problem  of  multibody  systems  in  space.  While 
solving  for  the  inverse  dynamics,  the  computations 
for  the  inverse  kinematics  are  considered  simulta¬ 
neously,  and  both  computations  arc  developed  on 
the  basis  of  momentum  constraints.  This  reduces  the 
computational  time  significantly.  We  establish  an 
efficient  algorithm  for  the  real  time  computation  of 
the  inverse  dynamics.  We  also  discuss  the  scope  of 
parallel  recursion  for  further  reduction  of  the  time 
for  computing  the  inverse  dynamics. 

Robotic  exploration  of  surfaces  and  its  application 
to  legged  locomotion 

93N13540 

SINIIA,  PRAMATH  RAJ  Avail:  Univ.  Microfilms 
Order  No.  DA92 12004 

Material  properties  like  penetrability,  compliance, 
and  surface  roughness  are  important  in  the  charac¬ 
terization  of  the  environment.  While  concentrating 
on  issues  of  geometry  and  shape,  researchers  in 
perceptual  robotics,  until  recently,  have  not  quite 
addressed  the  issue  of  the  extraction  of  material 
properties  from  the  environment.  The  goal  of  this 
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research  is  to  design  and  implement  a  robotic  sys¬ 
tem  that  will  actively  explore  a  surface  to  extract  its 
material  characteristics.  Further  the  relevance  of 
material  properties  in  the  legged  locomotion  of 
robots  is  also  recognized  and  our  research  objec¬ 
tives  are  extended  towards  building  a  robotic  system 
for  exploration  such  that  is  actively  perceives  mate¬ 
rial  properties  during  the  process  of  legged  locomo¬ 
tion.  The  chosen  approach  to  the  design  and  imple¬ 
mentation  of  such  a  robotic  system  is  to  first  select 
an  appropriate  environment  model  and  then  to  de¬ 
sign  exploratory  procedures  salient  to  each  attribute 
of  interest.  These  exploratory  procedures  are  then 
implemented  through  an  experimental  setup  and  the 
results  show  that  material  properties  can  be  reliably 
measured.  The  design,  implementation,  and  results 
of  a  framework  for  surface  exploration  to  recover 
material  properties  are  presented.  Further,  the  ex¬ 
ploratory  procedures  for  exploration  are  integrated 
into  an  active  perceptual  scheme  for  legged  locomo¬ 
tion.  The  perceptual  scheme  is  designed  around 
creating  the  ability  for  the  robot  to  sense  variations 
in  terrain  properties  while  it  is  walking,  so  that  it 
may  be  able  to  avoid  sinking,  slipping,  and  falling 
due  to  unexpected  changes  in  the  terrain  properties, 
and  make  suitable  changes  in  its  foot  forces  to  con¬ 
tinue  locomotion.  Finite  element  simulations  of  the 
foot-terrain  interaction  are  used  to  justify  some  of 
the  strategies  used  in  this  active  perceptual  scheme. 
The  active  perceptual  scheme  is  implemented  by 
simulating  a  leg-ankle-foot  system  with  a  PUMA 
arm-complaint  wrist-foot  system  and  an  accelerome¬ 
ter  mounted  on  the  foot  to  detect  slip.  Details  of 
implementation  and  experimental  results  are 
presented. 

The  3-D  computer  vision  techniques  for  object 
following  and  obstacle  avoidance 

93N13245 

EVANS,  RICHARD  In  AGARD,  Machine  Percep¬ 
tion  24  p  (SEE  N93- 13238  03-63)  Sponsored  in  part 
by  Commission  of  the  European  Communities 
Avail:  CASI  HC  A03/MF  A03 
Imaging  sensors  are  powerful  tools  enabling  remote 
control,  by  tele-operation,  of  numerous  tasks  where 
the  operator  requires  an  appreciation  of  the 
three-dimensional  structure  of  the  viewed  scene. 
Passive  video  sensors  also  lend  themselves  to  tasks 
where  covert  operation  or  electromagnetic  compati¬ 
bility  is  required.  A  commonly  mooted 
tele-operational  task  is  that  of  driving  a  known 
vehicle  through  an  unknown  terrain  -  or  keeping 
station  on  a  known  object  moving  through  an  un¬ 
known  terrain.  The  computer  vision  aspects  of  auto¬ 
mating  this  task  are  divided  into  two  separate  vision 
functions,  which  are  the  subjects  of  this  paper:  (1) 
analysis  of  image  sequences  of  a  general  scene  to 
extract  its  three  dimensional  (3D)  structure  without 
any  prior  information;  and  (2)  analysis  of  images  of 
a  well  defined  object,  to  extract  its  3D  position  and 
orientation  relative  to  the  sensor.  For  both  these 
functions,  the  paper  provides  a  brief  introduction  to 
possible  techniques  followed  by  further  descriptions 


of  particular  systems,  DROID  and  RAPiD,  devel¬ 
oped  by  Roke  Manor  Research  Limited.  DROID  is 
a  general,  feature-based  3D  vision  system  using  the 
structure-from-motion  principle.  That  is,  it  uses  the 
apparent  image-plane  movement  of  localized  fea¬ 
tures  viewed  by  a  moving  sensor  to  extract  the 
three-dimensional  structure  of  the  scene.  RAPiD  is 
a  model-based  real-time  tracker  which  extracts  the 
position  (X,Y,Z)  and  orientation  (roll,  pitch,  yaw)  of 
a  known  object  from  image  data.  The  system  oper¬ 
ates  iteratively,  using  prediction  of  object  pose  (po¬ 
sition  and  orientation)  to  cue  the  search  for  selected 
edge  features  in  subsequent  imagery.  This  approach 
results  in  minimal  processing  of  image  pixels,  so 
that  the  system  can  be  implemented  at  full  video 
rate  using  modest  hardware. 

Machine  perception  exploiting  high-level 
spatio-temporal  models 

93N 13244 

DICKMANNS,  E.  D.  In  AGARD,  Machine  Percep¬ 
tion  17  p  (SEE  N93-13238  03-63)  Avail:  CASI  HC 
A03/MF  A03 

A  paradigm  for  machine  perception  is  presented 
which  takes  time  and  3D  space  in  an  integrated 
manner  as  the  underlying  framework  for  internal 
representation  of  the  sensorially  observed  outside 
world.  This  world  is  considered  to  consist  of  mate¬ 
rial  and  mental  processes  evolving  over  time.  The 
concept  of  state  and  control  variables  developed  in 
the  natural  sciences  and  engineering  over  the  last 
three  centuries  is  exploited  to  find  a  new,  more 
natural  access  to  dynamic  real-time  vision  and  intel¬ 
ligence.  Schopenhauer's  conjecture  of ’The  world  as 
evolving  process  and  internal  representation'  (1819) 
is  combined  with  modem  recursive  estimation  tech¬ 
niques  (Kalman  60)  and  some  components  from 
geometry  and  AI  in  order  to  arrive  at  a  very  effi¬ 
cient  scheme  for  autonomous  robotic  agents  dealing 
with  evolving  processes  in  the  real  world  in  real 
time.  Application  to  autonomous  mobile  robots  is 
discussed. 

3-D  computer  vision  for  navigation/control  of 
mobile  robots 

93N 13243 

GARIBOTTO,  G.  B.;  MASCIANGELO,  S.  In 
AGARD,  Machine  Perception  13  p  (SEE  N93- 
13238  03-63)  Avail:  CASI  HC  A03/MF  A03 
The  aim  of  this  lecture  is  to  investigate  how  much 
visual  sensors  may  be  effective  in  supporting  au¬ 
tonomous  navigation  of  mobile  robots.  Although  in 
practical  realizations,  with  robustness  and  reliability 
constraints,  it  is  always  necessary  to  integrate  multi¬ 
sensor  modalities,  the  discussion  here  is  limited  to 
an  analysis  of  computer  vision  advantages  and  dis¬ 
advantages,  with  particular  attention  given  to:  (1)  a 
binocular  stereo  vision  module  for  obstacle  detec¬ 
tion,  with  no  precise  calibration  (reactive  process  to 
operate  at  a  fast  rate,  from  5  to  10  Hz);  trinocular 
stereovision  based  on  segment  primitives  for  the 
reconstruction  of  free  space  for  navigation,  in  which 
case  an  acccurate  calibration  procedure  is  requested; 
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and  landmark  detection  for  self-positioning  and 
orientation  of  a  mobile  vehicle,  using  perspective 
invariants,  for  indoor  navigation.  Some  comments 
are  also  provided  on  computer  vision  architectures 
to  support  real  time  implementations.  A  real-time 
front  end  vision  subsystem  is  described,  which  is 
able  to  compute  3D  segment  based  stereovision  at  5 
Hz  and  segment  token  tracking  at  10  Hz.  Finally, 
some  demo  arrangements  are  briefly  discussed 
where  an  intense  experimentation  of  such  results  is 
in  progress,  as  a  test  bed  for  different  industrial 
applications. 

Machine  Perception 

93N13238  Lecture  series  held  in  Harnton,  VA,  3-4 
Sep.  1992,  in  Neubiberg,  Germany,  14-15  Sep. 

1992,  and  in  Madrid,  Spain,  17-18  Sep.  1992 
Avail:  CASI  HC  A10/MF  A03 

Space  Station  robotics  planning  tools 

93N11975 

TESTA,  BRIDGET  MINTZ  In  NASA.  Lyndon  B. 
Johnson  Space  Center,  Fifth  Annual  Workshop  on 
Space  Operations  Applications  and  Research 
(SOAR  1991),  Volume  1  p  382-390  (SEE 
N93-1 1921  02-59)  Avail:  CASI  HC  A02/MF  A04 
The  concepts  are  described  for  the  set  of  advanced 
Space  Station  Freedom  (SSF)  robotics  planning 
tools  for  use  in  the  Space  Station  Control  Center 
(SSCC).  It  is  also  shown  how  planning  for  SSF 
robotics  operations  is  an  international  process,  and 
baseline  concepts  are  indicated  for  that  process. 
Current  SRMS  methods  provide  the  backdrop  for 
this  SSF  theater  of  multiple  robots,  long  operating 
time-space,  advanced  tools,  and  international  coop¬ 
eration. 

Remote  systems  development 

93N11971 

OLSEN,  R.;  SCHAEFER,  O.;  HUSSEY,  J.  In 
NASA.  Lyndon  B.  Johnson  Space  Center,  Fifth 
Annual  Workshop  on  Space  Operations  Applica¬ 
tions  and  Research  (SOAR  1991),  Volume  1  p 
331-347  (SEE  N93-1 1921  02-59)  Avail:  CASI  HC 
A03/MF  A04 

Potential  space  missions  of  the  nineties  and  the  next 
century  require  that  we  look  at  the  broad  category 
of  remote  systems  as  an  important  means  to  achieve 
cost-effective  operations,  exploration  and  coloniza¬ 
tion  objectives.  This  paper  addresses  such  missions, 
which  can  use  remote  systems  technology  as  the 
basis  for  identifying  required  capabilities  which 
must  be  provided.  The  relationship  of  the  space- 
based  tasks  to  similar  tasks  required  for  terrestrial 
applications  is  discussed.  The  development  status  of 
the  required  technology  is  assessed  and  major  issues 
which  must  be  addressed  to  meet  future  require¬ 
ments  are  identified.  This  includes  the  proper  mix 
of  humans  and  machines,  from  pure  teleoperation  to 
full  autonomy;  the  degree  of  worksite  compatibility 
for  a  robotic  system;  and  the  required  design  param¬ 
eters,  such  as  degrees-of-freedom.  Methods  for 
resolution  are  discussed  including  analysis,  graphi¬ 


cal  simulation  and  the  use  of  laboratory  test  beds. 
Grumman  experience  in  the  application  of  these 
techniques  to  a  variety  of  design  issues  are  present¬ 
ed  utilizing  the  Telerobotics  Development  Laborato¬ 
ry  which  includes  a  17-DOF  robot  system,  a  variety 
of  sensing  elements,  Deneb/IRIS  graphics 
workstations  and  control  stations.  The  use  of 
task/worksite  mockups,  remote  system  development 
test  beds  and  graphical  analysis  are  discussed  with 
examples  of  typical  results  such  as  estimates  of  task 
times,  task  feasibility  and  resulting  recommenda¬ 
tions  for  design  changes.  The  relationship  of  this 
experience  and  lessons-leamed  to  future  develop¬ 
ment  of  remote  systems  is  also  discussed. 

Exoskeleton  master  controller  with  force-reflecting 

telepresence 

93N11970 

BURKE,  JAMES  B.;  BARTHOLET,  STEPHEN  J.; 
NELSON,  DAVID  K.  (Aerospace  Medical  Research 
Labs.,  Wright-Patterson  AFB,  OH.)  In  NASA. 
Lyndon  B.  Johnson  Space  Center,  Fifth  Annual 
Workshop  on  Space  Operations  Applications  and 
Research  (SOAR  1991),  Volume  1  p  321-330  (SEE 
N93-1 1921  02-59)  Avail:  CASI  HC  A02/MF  A04 
A  thorough  understanding  of  the  requirements  for 
successful  master-slave  robotic  systems  is  becoming 
increasingly  desirable.  Such  systems  can  aid  in  the 
accomplishment  of  tasks  that  are  hazardous  or  inac¬ 
cessible  to  humans.  Although  a  history  of  use  has 
proven  master-slave  systems  to  be  viable,  system 
requirements  and  the  impact  of  specifications  on  the 
human  factors  side  of  system  performance  are  not 
well  known.  In  support  of  the  next  phase  of  tele¬ 
operation  research  being  conducted  at  the  Arm¬ 
strong  Research  Laboratory,  a  force-reflecting, 
seven  degree  of  freedom  exoskeleton  for  master- 
slave  teleoperation  has  been  concepted,  and  is  pres¬ 
ently  being  developed.  The  exoskeleton  has  a 
unique  kinematic  structure  that  complements  the 
structure  of  the  human  arm.  It  provides  a  natural 
means  for  teleoperating  a  dexterous,  possibly  redun¬ 
dant  manipulator.  It  allows  ease  of  use  without 
operator  fatigue  and  faithfully  follows  human  arm 
and  wrist  motions.  Reflected  forces  and  moments 
are  remotely  transmitted  to  the  operator  hand  grip 
using  a  cable  transmission  scheme.  This  paper  pres¬ 
ents  the  exoskeleton  concept  and  development 
results  to  date.  Conceptual  design,  hardware, 
algorithms,  computer  architecture,  and  software 
are  covered. 

The  development  of  system  components  to  provide 
proprioceptive  and  tactile  information  to  the  human 
for  future  telepresence  systems 

93N11968 

WRIGHT,  AMMON  K.  In  NASA.  Lyndon  B.  John¬ 
son  Space  Center,  Fifth  Annual  Workshop  on  Space 
Operations  Applications  and  Research  (SOAR 
1991),  Volume  1  p  311  (SEE  N93-1 1921  02-59) 
Avail:  CASI  HC  A01/MF  A04 
System  components  are  presented  that  are  being 
implemented  to  augment  teleoperated  systems  by 
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providing  both  force  and  tactile  information  to  the 
human  operator.  The  concept  proposed  is  the  con¬ 
trol  of  a  manipulator  to  perform  tasks;  i.e.,  flight 
line  maintenance  and  repair  of  combat  aircraft  or 
satellites  while  under  the  control  of  a  human  opera¬ 
tor  at  a  remote  location  to  maintain  mission  effec¬ 
tiveness  in  a  hostile  environment.  The  human  would 
control  the  motion  of  the  manipulator  via  a  master 
system  with  information  from  the  remote  site  being 
fed  back  by  direct  stimulation  of  the  humans  senso¬ 
ry  mechanisms  or  by  graphic  interpretation  of  dis¬ 
plays.  We  are  interested  in  providing  the  operator 
feedback  of  position,  force,  auditory,  vision,  and 
tactile  information  to  aide  in  the  human's  cognitive 
ability  to  control  the  manipulator.  This  sensory 
information  from  the  remote  site  would  then  be 
presented  to  the  operator  in  such  a  manner  as  to 
enhance  his  performance  while  providing  him  a 
sense  of  being  present  at  the  remote  location,  this  is 
known  as  telepresence.  Also  discussed  is  the  re¬ 
search  done  by  the  Human  Sensory  Feedback  (HSF) 
facility  at  the  Armstrong  Laboratory  to  provide 
tactile  and  proprioceptive  feedback  to  the  operator. 
The  system  components  of  this  system  includes 
tactile  sensor  and  stimulators,  dexterous  robotic 
hands,  and  the  control  of  positioning  and  operating 
industrial  robots  with  exoskeletal  mechanisms. 

Effects  of  spatially  displaced  feedback  on  remote 
manipulation  tasks 

93N11966 

MANAHAN,  MEERA  K.;  STUART,  MARK  A.; 
BIERSCHWALE,  JOHN  M.;  HWANG,  ELLEN  Y.; 
LEGENDRE,  A.  J.  (Lockheed  Engineering  and 
Sciences  Co.,  Houston,  TX.);  (Lockheed  Engineer¬ 
ing  and  Sciences  Co.,  Houston,  TX.);  (Lockheed 
Engineering  and  Sciences  Co.,  Houston,  TX.); 
(Lockheed  Engineering  and  Sciences  Co.,  Houston, 
TX.)  In  its  Fifth  Annual  Workshop  on  Space  Opera¬ 
tions  Applications  and  Research  (SOAR  1991), 
Volume  1  p  302-309  (SEE  N93-1 1921  02-59) 

Avail:  CASI  HC  A02/MF  A04 
Several  studies  have  been  performed  to  determine 
the  effects  on  computer  and  direct  manipulation  task 
performance  when  viewing  conditions  are  spatially 
displaced.  Whether  results  from  these  studies  can  be 
directly  applied  to  remote  manipulation  tasks  is 
quenstionable.  The  objective  of  this  evaluation  was 
to  determine  the  effects  of  reversed,  inverted,  and 
inverted/reversed  views  on  remote  manipulation  task 
performance  using  two  3-Degree  of  Freedom  (DOF) 
hand  controllers  and  a  replica  position  hand  control¬ 
ler.  Results  showed  that  trials  using  the  inverted 
viewing  condition  showed  the  worst  performance, 
followed  by  the  inverted/reversed  view  and  the 
reversed  view  when  using  the  2x3  DOF.  However, 
these  differences  were  not  significant.  The  inverted 
and  inverted/reversed  viewing  conditions  were  sig¬ 
nificantly  worse  than  the  normal  and  reversed  view¬ 
ing  conditions  when  using  the  Kraft  Replica.  A 
second  evaluation  was  conducted  in  which  addition¬ 
al  trials  were  performed  with  each  viewing  condi¬ 
tion  to  determine  the  long  term  effects  of  spatially 


displaced  views  on  task  performance  for  the  hand 
controllers.  Results  of  the  second  evaluation  indicat¬ 
ed  that  there  was  more  of  a  difference  in  perfor¬ 
mance  between  the  perturbed  viewing  conditions 
and  the  normal  viewing  condition  with  the  Kraft 
Replica  than  with  the  2x3  DOF. 

A  neuro-collision  avoidance  strategy  for  robot 
manipulators 

93N 11962 

ONEMA,  JOEL  P.;  MACLAUNCHLAN,  ROBERT 
A.  In  NASA.  Lyndon  B.  Johnson  Space  Center, 

Fifth  Annual  Workshop  on  Space  Operations  Appli¬ 
cations  and  Research  (SOAR  1991),  Volume  1  p 
272-279  (SEE  N93- 11921  02-59)  Avail:  CASI  HC 
A02/MF  A04 

The  area  of  collision  avoidance  and  path  planning 
in  robotics  has  received  much  attention  in  the  re¬ 
search  community.  Our  study  centers  on  a  combi¬ 
nation  of  an  artificial  neural  network  paradigm  with 
a  motion  planning  strategy  that  insures  safe  motion 
of  the  Articulated  Two-Link  Arm  with  Scissor  Hand 
System  relative  to  an  object.  Whenever  an  obstacle 
is  encountered,  the  arm  attempts  to  slide  along  the 
obstacle  surface,  thereby  avoiding  collision  by 
means  of  the  local  tangent  strategy  and  its  artificial 
neural  network  implementation.  This  combination 
compensates  the  inverse  kinematics  of  a  robot 
manipulator.  Simulation  results  indicate  that  a 
neuro-collision  avoidance  strategy  can  be 
achieved  by  means  of  a  learning  local  tangent 
method. 

A  new  scheme  of  force  reflecting  control 

93N11960 

KIM,  WON  S.  In  NASA.  Lyndon  B.  Johnson  Space 
Center,  Fifth  Annual  Workshop  on  Space  Opera¬ 
tions  Applications  and  Research  (SOAR  1991), 
Volume  1  p  254-261  (SEE  N93-11921  02-59)  Spon¬ 
sored  by  NASA,  Washington  Avail:  CASI  HC 
A02/MF  A04 

A  new  scheme  of  force  reflecting  control  has  been 
developed  that  incorporates  position-error-based 
force  reflection  and  robot  compliance  control.  The 
operator  is  provided  with  a  kinesthetic  force  feed¬ 
back  which  is  proportional  to  the  position  error 
between  the  operator-commanded  and  the  actual 
position  of  the  robot  arm.  Robot  compliance  con¬ 
trol,  which  increases  the  effective  compliance  of  the 
robot,  is  implemented  by  low  pass  filtering  the 
outputs  of  the  force/torque  sensor  mounted  on  the 
base  of  robot  hand  and  using  these  signals  to  alter 
the  operator's  position  command.  This  position- 
error-based  force  reflection  scheme  combined  with 
shared  compliance  control  has  been  implemented 
successfully  to  the  Advanced  Teleoperation  system 
consisting  of  dissimilar  master-slave  arms.  Stability 
measurements  have  demonstrated  unprecedentedly 
high  force  reflection  gains  of  up  to  2  or  3,  even 
though  the  slave  arm  is  much  stiffer  than  operator's 
hand  holding  the  force  reflecting  hand  controller. 
Peg-in-hole  experiments  were  performed  with  eight 
different  operating  modes  to  evaluate  the  new 
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force-reflecting  control  scheme.  Best  task  perfor¬ 
mance  resulted  with  this  new  control  scheme. 

Fuzzy  logic  control  of  telerobot  manipulators 

93N 11959 

FRANKE,  ERNEST  A.;  NEDUNGADI,  ASHOK  In 
NASA.  Lyndon  B.  Johnson  Space  Center,  Fifth 
Annual  Workshop  on  Space  Operations  Applica¬ 
tions  and  Research  (SOAR  1991),  Volume  1  p 
248-253  (SEE  N93-1 1921  02-59)  Avail:  CASI  HC 
A02/MF  A 04 

Telerobot  systems  for  advanced  applications  will 
require  manipulators  with  redundant  'degrees  of 
freedom'  (DOF)  that  are  capable  of  adapting  ma¬ 
nipulator  configurations  to  avoid  obstacles  while 
achieving  the  user  specified  goal.  Conventional 
methods  for  control  of  manipulators  (based  on  solu¬ 
tion  of  the  inverse  kinematics)  cannot  be  easily 
extended  to  these  situations.  Fuzzy  logic  control 
offers  a  possible  solution  to  these  needs.  A  current 
research  program  at  SRI  developed  a  fuzzy  logic 
controller  for  a  redundant,  4  DOF,  planar  manipu¬ 
lator.  The  manipulator  end  point  trajectory  can  be 
specified  by  either  a  computer  program  (robot 
mode)  or  by  manual  input  (teleoperator).  The  ap¬ 
proach  used  expresses  end-point  error  and  the  loca¬ 
tion  of  manipulator  joints  as  fuzzy  variables.  Joint 
motions  are  determined  by  a  fuzzy  rule  set  without 
requiring  solution  of  the  inverse  kinematics.  Addi¬ 
tional  rules  for  sensor  data,  obstacle  avoidance  and 
preferred  manipulator  configuration,  e.g.,  'righty'  or 
'lefty',  are  easily  accommodated.  The  procedure 
used  to  generate  the  fuzzy  rules  can  be  extended  to 
higher  DOF  systems. 

Contact  detection  and  contact  motion  for  error 
recovery  in  the  presence  of  uncertainties 

93N11956 

XIAO,  JING  In  NASA.  Lyndon  B.  Johnson  Space 
Center,  Fifth  Annual  Workshop  on  Space  Opera¬ 
tions  Applications  and  Research  (SOAR  1991), 
Volume  1  p  230-237  (SEE  N93-11921  02-59) 

Avail:  CASI  HC  A02/MF  A04 
Due  to  various  kinds  of  uncertainties,  a  robot  mo¬ 
tion  may  fail  and  result  in  some  unintended  contact 
between  the  object  held  by  the  robot  and  the  envi¬ 
ronment,  which  greatly  hampers  robotics  applica¬ 
tions  on  tasks  with  high-precision  requirements, 
such  as  assembly  tasks.  Aiming  at  automatically 
recovering  a  robotic  task  from  such  a  failure,  this 
paper  discusses,  in  the  presence  of  uncertainties, 
contact  detection  based  on  contact  motion  for  re¬ 
covery.  It  presents  a  framework  for  on-line  recog¬ 
nizing  contacts  using  multiple  sensor  modalities  in 
the  presence  of  sensing  uncertainties  and  means  for 
ensuring  successful  compliant  motions  in  the  pres¬ 
ence  of  sensing  and  control  uncertainties. 

Requirements  and  applications  for  robotic  servicing 
of  military  space  systems 

93N11953 

LEDFORD,  OTTO  C.,  JR.;  BENNETT,  RODNEY 
G.  (Air  Force  Space  Div.,  El  Segundo,  CA.)  In 


NASA.  Lyndon  B.  Johnson  Space  Center,  Fifth 
Annual  Workshop  on  Space  Operations  Applica¬ 
tions  and  Research  (SOAR  1991),  Volume  1  p 
206-212  (SEE  N93-1 1921  02-59)  Avail:  CASI  HC 
A02/MF  A04 

The  utility  of  on-orbit  servicing  of  spacecraft  has 
been  demonstrated  by  NASA  several  times  using 
shuttle-based  astronaut  EVA.  There  has  been  inter¬ 
est  in  utilizing  on-orbit  servicing  for  military  space 
systems  as  well.  This  interest  has  been  driven  by 
the  increasing  reliance  of  all  branches  of  the  mili¬ 
tary  upon  space-based  assets,  the  growing  numbers, 
complexity,  and  cost  of  those  assets,  and  a  desire  to 
normalize  support  policies  for  space-based  opera¬ 
tions.  Many  military  satellites  are  placed  in  orbits 
which  are  unduly  hostile  for  astronaut  operations 
and/or  cannot  be  reached  by  the  shuttle.  In  addition, 
some  of  the  projected  tasks  may  involve  hazardous 
operations.  This  has  led  to  a  focus  on  robotic  sys¬ 
tems,  instead  of  astronauts,  for  the  basis  of  project¬ 
ed  servicing  systems.  This  paper  describes  studies 
and  activities  which  will  hopefully  lead  to  on-orbit 
servicing  being  one  of  the  tools  available  to  military 
space  systems  designers  and  operators.  The  utility 
of  various  forms  of  servicing  has  been  evaluated  for 
present  and  projected  systems,  critical  technologies 
have  been  identified,  and  strategies  for  the  develop¬ 
ment  and  insertion  of  this  technology  into  opera¬ 
tional  systems  have  been  developed.  Many  of  the 
projected  plans  have  been  adversely  affected  by 
budgetary  restrictions  and  evolving  architectures, 
but  the  fundamental  benefits  and  requirements  are 
well  understood.  A  method  of  introducing  servicing 
capabilities  in  a  manner  which  has  a  low  impact  on 
the  system  designer  and  does  not  require  the  prior 
development  of  an  expensive  infrastructure  is  dis¬ 
cussed.  This  can  potentially  lead  to  an  evolutionary 
implementation  of  the  full  technology. 

Integration  of  task  level  planning  and  diagnosis  for 
an  intelligent  robot 

93N11929 

CHAN,  AMY  W.  In  NASA.  Lyndon  B.  Johnson 
Space  Center,  Fifth  Annual  Workshop  on  Space 
Operations  Applications  and  Research  (SOAR 
1991),  Volume  1  p  52-59  (SEE  N93-11921  02-59) 
Avail:  CASI  HC  A02/MF  A04 
A  satellite  floating  space  is  diagnosed  with  a 
telerobot  attached  performing  maintenance  or  re¬ 
placement  tasks.  This  research  included  three  ob¬ 
jectives.  The  first  objective  was  to  generate  intelli¬ 
gent  path  planning  for  a  robot  to  move  around  a 
satellite.  The  second  objective  was  to  diagnose 
possible  faulty  scenarios  in  the  satellite.  The  third 
objective  included  two  tasks.  The  first  task  was  to 
combine  intelligent  path  planning  with  diagnosis. 
The  second  task  was  to  build  an  interface  between 
the  combined  intelligent  system  with  Robosim.  The 
ability  of  a  robot  to  deal  with  unexpected  scenarios 
is  particularly  important  in  space  since  the  situation 
could  be  different  from  time  to  time  so  that  the 
telerobot  must  be  capable  of  detecting  that  the  situa¬ 
tion  has  changed  and  the  necessity  may  exist  to 
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alter  its  behavior  based  on  the  new  situation.  The 
feature  of  allowing  human-in-the-loop  is  also  very 
important  in  space.  In  some  extreme  cases,  the 
situation  is  beyond  the  capability  of  a  robot  so  our 
research  project  allows  the  human  to  override  the 
decision  of  a  robot. 

ALECSYS  and  the  AutonoMouse:  Learning  to 
control  a  real  robot  by  distributed  classifier  systems 

93N11227 

DORIGO,  MARCO  Avail:  Politecnico  di  Milano, 
Piazza  Leonardo  da  Vinci  32,  20133  Milan,  Italy 
ALECSYS  (A  LEaming  Classifier  SYStem),  is  a 
parallel  distributed  learning  classifier  system  that 
provides  the  learning  system  designer  with  two 
basic  tools.  The  first  tool,  low  level  parallelism, 
allows  him  to  use  the  desired  computational  power 
by  parallelization  of  a  single  learning  Classifier 
System  (CS);  the  second  tool,  high  level  parallel¬ 
ism,  gives  him  the  possibility  to  connect,  with  great 
flexibility,  many  cooperating  CS's.  AutonoMouse,  a 
little  autonomous  robot,  shaped  like  a  mouse,  that 
gives  ALECSYS  a  body,  is  discussed.  The  design 
of  autonomous  mouse  sized  robots  (AutonoMouses) 
with  the  capacity  of  surviving  in  a  real  environment 
is  focused  upon.  The  overall  goal  is  very  ambitious 
and  cannot  be  solved  by  any  existing  computational 
paradigm.  Results  obtained  using  ALECSYS  as 
learning  software,  the  transputer  as  computing  hard¬ 
ware,  and  a  home  made  AutonoMouse  as  robot,  are 
reported.  ALECSYS  is  illustrated  and  experiments, 
both  in  a  simulated  environment  and  in  the  real 
world,  are  reported. 

Selective  perception  for  robot  driving 

93N10749 

REECE,  DOUGLAS  A.  Avail:  CASI  HC  A 10/ 

MF  A03 

Robots  performing  complex  tasks  in  rich  environ¬ 
ments  need  very  good  perception  modules  in  order 
to  understand  their  situation  and  choose  the  best 
action.  Robot  planning  systems  have  typically  as¬ 
sumed  that  perception  was  so  good  that  it  could 
refresh  the  entire  world  model  whenever  the  plan¬ 
ning  system  needed  it,  or  whenever  anything  in  the 
world  changed.  Unfortunately,  this  assumption  is 
completely  unrealistic  in  many  real-world  domains 
because  perception  is  far  too  difficult.  Robots  in 
these  domains  cannot  use  the  traditional  planner 
paradigm,  but  instead  need  a  new  system  design 
that  integrates  reasoning  with  perception.  In  this 
thesis  I  describe  how  reasoning  can  be  integrated 
with  perception,  how  task  knowledge  can  be  used  to 
select  perceptual  targets,  and  how  this  selection 
dramatically  reduces  the  computational  cost  of  per¬ 
ception.  The  domain  addressed  in  this  thesis  is 
driving  in  traffic.  I  have  developed  a  microscopic 
traffic  simulator  called  PHAROS  that  defines  the 
street  environment  for  this  research.  PHAROS  con¬ 
tains  detailed  representations  of  streets,  markings, 
signs,  signals,  and  cars.  It  can  simulate  perception 
and  implement  commands  for  a  vehicle  controlled 
by  a  separate  program.  I  have  also  developed  a 


computational  model  of  driving  called  Ulysses  that 
defines  the  driving  task.  The  model  describes  how 
various  traffic  objects  in  the  world  determine  what 
actions  that  a  robot  must  take. 

Optimal  robot  trajectory  planning 

93N10570 

BUCHAL,  R.  O.  In  Carleton  Univ.,  Proceedings  of 
the  Twelfth  Canadian  Congress  of  Applied  Mechan¬ 
ics,  Volumes  1  and  2  p  858-859  (SEE  N93-10466 
01-31)  Avail:  Issuing  Activity  (Canadian  Society 
for  Mechanical  Engineering,  2050  Mansfield  St., 
Suite  700,  Montreal,  Quebec  H3A  1Z7  Canada) 

In  order  to  achieve  the  objectives  of  off-line  robot 
programming  and  automatic  program  synthesis,  an 
effective  trajectory  planning  method  is  required.  A 
method  is  presented  for  determining  optimal  robot 
trajectories  subject  to  the  constraints  of  collision 
avoidance,  joint  displacement  limits,  and  joint  actu¬ 
ator  force  limits.  The  constraints  are  represented  by 
penalty  functions,  and  a  discrete  gradient  approach 
is  used  to  optimize  the  equivalent  unconstrained 
objective  functional.  The  model  includes  the  dynam¬ 
ic  equations  of  motion,  which  were  generated  for 
the  planar  four-link  manipulator  using  the 
MACSYMA  symbolic  algebra  package.  In  the  cur¬ 
rent  implementation,  the  user  interactively  specifies 
a  number  of  different  starting  trajectories  so  that 
several  alternative  locally  optimal  trajectories  are 
generated. 
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14.  Abstract 

To  ensure  the  capability  of  defence,  a  demand  for  equipment  and  systems  which  can  be  embraced 
under  the  title  of  “Robotics”  will  emerge  in  the  near  future.  In  this  context,  “Robotics”  represents  a 
specific  problem  area  involving  all  the  guidance  and  control  functions  which  are  associated  with 
achieving  goal-oriented  autonomous  behaviour  in  structured  and  unstructured  environments  for 
mobile  and  manipulator  systems  as  applied  to  ground,  sea,  air  and  space  operations.  Related 
robotic  systems  must  combine  constituent  functions  such  as  intelligent  decision  making,  control, 
manipulation,  motion,  sensing  and  communication. 

The  scope  of  the  special  course  will  cover  new  developments  in  the  areas  of  autonomous 
navigation  for  planetary  and  surface  systems,  and  control  and  operations  of  remote  manipulators. 

Topics  to  be  covered  include: 

•  Kinematics,  dynamics  and  mobility; 

•  Sensing  (vision,  tactile,  acoustic,  etc.)  and  sensory  processing; 

•  Sensory  interactive  task  decomposition,  planning  and  problem  solving; 

•  World  modelling; 

•  Programming  techniques  and  learning,  cognitive  control,  adaptive  sensory-motor 
control; 

•  System  integration,  test  and  evaluation; 

•  Man-machine  interfaces. 
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